2024-09-17T23:18:43.307 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T23:18:43.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T23:18:43.413 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909589' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLs8FF+xIy01YbyotcQ123T663AWzxzrKj5oV2BGsrHKszR17MqVQEBcYvHpx+oMEqirCAlL2z1AJXbhHXzIxqY= smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKs3Rm/2/C2sFChM6hV2e2V6ZuJcSxS7U0MysWAL+UuqNlrofYCPM9oCemhK8JQlG+0VN2uUps68eK4axEJBlXQ= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgohmViWyCpuOwR5MJ8irE3Q36uxhMFz42p4fqWgsKZ5YZexH+y1QU53MPUGPG9xC1u1m2/W9Zr0Mgp41VmRM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T23:18:43.414 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-17T23:18:43.414 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-17T23:18:43.415 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T23:18:43.416 INFO:teuthology.task.internal:Checking packages... 2024-09-17T23:18:43.438 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-17T23:18:43.438 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T23:18:43.438 INFO:teuthology.packaging:ref: None 2024-09-17T23:18:43.438 INFO:teuthology.packaging:tag: None 2024-09-17T23:18:43.439 INFO:teuthology.packaging:branch: squid 2024-09-17T23:18:43.439 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:18:43.439 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-17T23:18:43.655 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-17T23:18:43.657 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T23:18:43.663 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T23:18:43.663 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T23:18:43.685 INFO:teuthology.task.internal:Saving configuration 2024-09-17T23:18:43.697 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T23:18:43.704 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T23:18:43.727 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 23:13:46.059780', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLs8FF+xIy01YbyotcQ123T663AWzxzrKj5oV2BGsrHKszR17MqVQEBcYvHpx+oMEqirCAlL2z1AJXbhHXzIxqY='} 2024-09-17T23:18:43.745 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 23:13:46.061294', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKs3Rm/2/C2sFChM6hV2e2V6ZuJcSxS7U0MysWAL+UuqNlrofYCPM9oCemhK8JQlG+0VN2uUps68eK4axEJBlXQ='} 2024-09-17T23:18:43.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 23:13:46.062559', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgohmViWyCpuOwR5MJ8irE3Q36uxhMFz42p4fqWgsKZ5YZexH+y1QU53MPUGPG9xC1u1m2/W9Zr0Mgp41VmRM='} 2024-09-17T23:18:43.760 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T23:18:43.766 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-17T23:18:43.766 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-17T23:18:43.766 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['client.0'] 2024-09-17T23:18:43.766 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T23:18:43.839 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f47d2ddfac0>, signals=[15]) 2024-09-17T23:18:43.839 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T23:18:43.846 INFO:teuthology.task.internal:Opening connections... 2024-09-17T23:18:43.846 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:18:43.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:18:43.922 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:18:43.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:18:43.997 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-17T23:18:43.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:18:44.072 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T23:18:44.078 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-17T23:18:44.096 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-17T23:18:44.097 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-17T23:18:44.153 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-17T23:18:44.153 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-17T23:18:44.153 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-17T23:18:44.153 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-17T23:18:44.153 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T23:18:44.154 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T23:18:44.155 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-17T23:18:44.181 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-09-17T23:18:44.199 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-09-17T23:18:44.199 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-09-17T23:18:44.257 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-09-17T23:18:44.257 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-09-17T23:18:44.257 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-09-17T23:18:44.257 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-09-17T23:18:44.257 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-09-17T23:18:44.257 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T23:18:44.258 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T23:18:44.258 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-09-17T23:18:44.272 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-17T23:18:44.291 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-17T23:18:44.291 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-17T23:18:44.349 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T23:18:44.350 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T23:18:44.351 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T23:18:44.351 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-17T23:18:44.381 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T23:18:44.389 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T23:18:44.395 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T23:18:44.395 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-17T23:18:44.398 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-09-17T23:18:44.400 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-17T23:18:44.417 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T23:18:44.424 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T23:18:44.424 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-17T23:18:44.455 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-09-17T23:18:44.458 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-17T23:18:44.487 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T23:18:44.558 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T23:18:44.590 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T23:18:44.590 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T23:18:44.590 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T23:18:44.591 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T23:18:44.591 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T23:18:44.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:18:44.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:18:44.609 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-17T23:18:44.610 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-17T23:18:44.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:18:44.611 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-17T23:18:44.667 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-17T23:18:44.667 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-17T23:18:44.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-17T23:18:44.668 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-503.el9.x86_64 2024-09-17T23:18:44.668 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-503.el9.x86_64 2024-09-17T23:18:44.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-09-17T23:18:44.669 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-503.el9.x86_64 2024-09-17T23:18:44.669 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-503.el9.x86_64 2024-09-17T23:18:44.670 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-09-17T23:18:47.058 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-17T23:18:47.252 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-17T23:18:48.544 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.3 MB 00:02 2024-09-17T23:18:48.859 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-17T23:18:49.145 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-17T23:18:51.712 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-17T23:18:53.863 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.5 MB 00:00 2024-09-17T23:18:55.247 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 131 kB/s | 19 kB 00:00 2024-09-17T23:18:55.259 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-17T23:18:56.843 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 19 kB 00:00 2024-09-17T23:18:57.142 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 13 MB/s | 23 MB 00:01 2024-09-17T23:18:58.480 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-17T23:18:59.603 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.5 MB 00:03 2024-09-17T23:19:01.287 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 19 kB 00:00 2024-09-17T23:19:02.682 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-17T23:19:03.066 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-17T23:19:03.878 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-17T23:19:04.622 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T23:19:04.622 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T23:19:04.725 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T23:19:04.745 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:Install 4 Packages 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:19:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 79 M 2024-09-17T23:19:04.747 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 126 M 2024-09-17T23:19:04.747 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:19:04.925 INFO:teuthology.orchestra.run.smithi037.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 307 kB/s | 36 kB 00:00 2024-09-17T23:19:05.692 INFO:teuthology.orchestra.run.smithi037.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-17T23:19:05.776 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T23:19:05.776 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T23:19:05.860 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:19:05.869 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:19:05.869 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-17T23:19:05.869 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-17T23:19:05.870 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:19:05.871 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 79 M 2024-09-17T23:19:05.871 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 126 M 2024-09-17T23:19:05.871 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:19:06.192 INFO:teuthology.orchestra.run.smithi037.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-17T23:19:06.271 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 299 kB/s | 36 kB 00:00 2024-09-17T23:19:06.500 INFO:teuthology.orchestra.run.smithi037.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-17T23:19:06.501 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:19:06.501 INFO:teuthology.orchestra.run.smithi037.stdout:Total 45 MB/s | 79 MB 00:01 2024-09-17T23:19:07.210 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:19:07.614 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:19:07.614 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:19:08.585 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:19:08.585 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:19:08.613 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 6.4 MB/s | 16 MB 00:02 2024-09-17T23:19:08.679 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-17T23:19:10.602 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T23:19:10.603 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T23:19:10.696 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:19:10.711 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:19:10.711 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-17T23:19:10.712 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:19:10.713 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-17T23:19:10.713 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-17T23:19:10.713 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:19:10.822 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 407 kB/s | 36 kB 00:00 2024-09-17T23:19:10.995 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 7.1 MB/s | 35 MB 00:04 2024-09-17T23:19:11.146 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:19:11.529 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.4 MB/s | 29 MB 00:05 2024-09-17T23:19:11.529 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:19:11.530 INFO:teuthology.orchestra.run.smithi139.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-17T23:19:11.820 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T23:19:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:19:11.914 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-17T23:19:12.211 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:19:12.516 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 19 MB/s | 35 MB 00:01 2024-09-17T23:19:12.568 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:19:12.568 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:19:12.833 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 14 MB/s | 29 MB 00:02 2024-09-17T23:19:12.834 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:19:12.834 INFO:teuthology.orchestra.run.smithi031.stdout:Total 37 MB/s | 79 MB 00:02 2024-09-17T23:19:13.517 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:19:13.517 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:19:13.549 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:19:13.929 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:19:13.929 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:19:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:19:14.771 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:19:14.949 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:19:14.949 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:19:15.945 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:19:16.758 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T23:19:16.845 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:19:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:19:18.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T23:19:18.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:19:19.596 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:19:19.724 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:19:19.899 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:19:19.939 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:19:21.001 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:19:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:19:24.584 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:19:24.715 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:19:25.077 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:19:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:19:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:19:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:19:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:04.093 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:04.093 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T23:20:04.093 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:20:04.093 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:20:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:20:04.935 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:20:04.935 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T23:20:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T23:20:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T23:20:04.936 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:20:04.936 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:20:05.301 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T23:20:05.387 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:05.387 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T23:20:05.387 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T23:20:05.387 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T23:20:05.387 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T23:20:05.388 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T23:20:05.388 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T23:20:05.388 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T23:20:05.447 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:05.447 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T23:20:05.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-09-17T23:20:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 82.el9 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8274670 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T23:20:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:20:05.480 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T23:20:05.481 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T23:20:05.481 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 5.14.0-508.el9.x86_64 2024-09-17T23:20:05.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T23:20:06.097 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-09-17T23:20:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T23:20:07.344 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-09-17T23:20:07.346 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-17T23:20:07.377 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.6OmQoKiyV6 2024-09-17T23:20:07.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6OmQoKiyV6 2024-09-17T23:20:07.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.6OmQoKiyV6 2024-09-17T23:20:07.584 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.6OmQoKiyV6 is 6KB 2024-09-17T23:20:07.633 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.6OmQoKiyV6 2024-09-17T23:20:07.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T23:20:07.713 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T23:20:07.713 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T23:20:07.713 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T23:20:07.714 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T23:20:07.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T23:20:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:07.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-09-17T23:20:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:08.159 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:08.159 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T23:20:08.159 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:20:08.159 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:20:08.994 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:20:08.995 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:20:09.394 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T23:20:09.483 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:09.484 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T23:20:09.484 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T23:20:09.484 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T23:20:09.484 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T23:20:09.484 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-17T23:20:09.484 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T23:20:09.485 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T23:20:09.543 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:09.544 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T23:20:09.544 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-09-17T23:20:09.583 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 82.el9 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8274670 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T23:20:09.584 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:20:09.585 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T23:20:09.586 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T23:20:09.586 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-508.el9.x86_64 2024-09-17T23:20:09.586 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T23:20:10.244 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-09-17T23:20:10.687 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:10.729 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T23:20:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T23:20:11.042 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T23:20:11.566 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T23:20:11.598 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-09-17T23:20:11.600 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-17T23:20:11.616 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.yEubUjkKSt 2024-09-17T23:20:11.616 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yEubUjkKSt 2024-09-17T23:20:11.685 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.yEubUjkKSt 2024-09-17T23:20:11.824 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.yEubUjkKSt is 6KB 2024-09-17T23:20:11.872 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.yEubUjkKSt 2024-09-17T23:20:11.887 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T23:20:11.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:20:11.898 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:20:11.950 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T23:20:11.951 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T23:20:11.951 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T23:20:11.951 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T23:20:11.952 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T23:20:12.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-09-17T23:20:12.237 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T23:20:12.328 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:12.329 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T23:20:12.329 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T23:20:12.329 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T23:20:12.329 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T23:20:12.329 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T23:20:12.329 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T23:20:12.329 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T23:20:12.388 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T23:20:12.388 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T23:20:12.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-17T23:20:12.419 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-17T23:20:12.420 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-17T23:20:12.420 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-17T23:20:12.420 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-17T23:20:12.420 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-17T23:20:12.420 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T23:20:12.421 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-17T23:20:12.421 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-17T23:20:12.421 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-17T23:20:12.424 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T23:20:12.424 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T23:20:12.424 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T23:20:12.424 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T23:20:12.424 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-17T23:20:12.425 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-17T23:20:12.425 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T23:20:12.425 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-17T23:20:12.425 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-17T23:20:12.425 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:20:12.425 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T23:20:12.426 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T23:20:12.426 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T23:20:12.426 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-17T23:20:12.426 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:20:12.426 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T23:20:12.428 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T23:20:12.428 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-508.el9.x86_64 2024-09-17T23:20:12.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T23:20:13.047 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-17T23:20:14.381 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T23:20:14.401 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-17T23:20:14.404 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-17T23:20:14.419 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.3VU1heDpPv 2024-09-17T23:20:14.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3VU1heDpPv 2024-09-17T23:20:14.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.3VU1heDpPv 2024-09-17T23:20:14.636 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.3VU1heDpPv is 6KB 2024-09-17T23:20:14.684 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.3VU1heDpPv 2024-09-17T23:20:14.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T23:20:14.768 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T23:20:14.768 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T23:20:14.768 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T23:20:14.768 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T23:20:14.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T23:20:14.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-17T23:20:37.881 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T23:20:37.882 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T23:20:37.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:20:42.093 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T23:20:42.093 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:20:42.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:20:44.929 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T23:20:44.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-17T23:20:44.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:21:00.477 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-17T23:21:09.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:21:09.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:21:12.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-17T23:21:24.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:21:24.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:21:37.894 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T23:21:42.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-17T23:21:44.933 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T23:21:46.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T23:21:46.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:21:47.392 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-17T23:21:48.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T23:21:48.047 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T23:21:48.048 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-17T23:21:48.063 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-508.el9.x86_64 2024-09-17T23:21:48.063 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T23:21:48.063 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T23:21:48.063 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T23:21:49.065 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T23:21:49.065 DEBUG:teuthology.orchestra.run.smithi037:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T23:21:49.144 INFO:teuthology.orchestra.run.smithi037.stdout:ttyS1 2024-09-17T23:21:49.169 DEBUG:teuthology.parallel:result is None 2024-09-17T23:21:53.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-17T23:21:53.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:21:54.497 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-17T23:21:54.842 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-17T23:21:54.842 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T23:21:54.842 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-17T23:21:54.857 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-508.el9.x86_64 2024-09-17T23:21:54.857 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T23:21:54.858 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T23:21:54.858 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T23:21:55.858 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T23:21:55.859 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T23:21:55.929 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-17T23:21:55.955 DEBUG:teuthology.parallel:result is None 2024-09-17T23:21:57.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:21:57.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:21:58.503 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-17T23:21:58.918 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:21:58.918 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-17T23:21:58.918 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-17T23:21:58.933 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-508.el9.x86_64 2024-09-17T23:21:58.933 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T23:21:58.933 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T23:21:58.933 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T23:21:59.934 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-17T23:21:59.934 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T23:22:00.072 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-09-17T23:22:00.093 DEBUG:teuthology.parallel:result is None 2024-09-17T23:22:00.094 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T23:22:00.101 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T23:22:00.101 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T23:22:00.104 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T23:22:00.106 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T23:22:00.144 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T23:22:00.151 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T23:22:00.160 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T23:22:00.160 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T23:22:00.163 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T23:22:00.166 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T23:22:00.245 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T23:22:00.251 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T23:22:00.251 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T23:22:00.254 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T23:22:00.257 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T23:22:00.289 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T23:22:00.294 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T23:22:00.302 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T23:22:00.306 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T23:22:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T23:22:00.359 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T23:22:00.362 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T23:22:00.369 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T23:22:00.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T23:22:00.371 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T23:22:00.374 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T23:22:00.433 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T23:22:00.443 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T23:22:00.443 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T23:22:00.446 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T23:22:00.448 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T23:22:00.489 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T23:22:00.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T23:22:00.598 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T23:22:00.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T23:22:00.727 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:22:00.727 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T23:22:00.797 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T23:22:00.842 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T23:22:00.899 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T23:22:00.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T23:22:01.028 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:22:01.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T23:22:01.096 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T23:22:01.133 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T23:22:01.189 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T23:22:01.264 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T23:22:01.311 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:22:01.311 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T23:22:01.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-17T23:22:01.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-09-17T23:22:01.380 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-17T23:22:01.423 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:22:01.425 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:22:01.462 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:22:01.986 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T23:22:01.994 INFO:teuthology.task.internal:Starting timer... 2024-09-17T23:22:01.994 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T23:22:02.033 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T23:22:02.104 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-17T23:22:02.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-17T23:22:02.163 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-17T23:22:02.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-09-17T23:22:02.216 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-09-17T23:22:02.219 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-09-17T23:22:02.266 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-09-17T23:22:02.268 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T23:22:02.268 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-17T23:22:02.441 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-17T23:22:02.441 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-09-17T23:22:02.462 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-09-17T23:22:02.462 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-09-17T23:22:02.505 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-09-17T23:22:02.505 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2024-09-17T23:22:02.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:22:02.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:22:02.536 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:22:02.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:22:02.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:22:02.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:22:02.591 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T23:22:02.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-17T23:22:02.621 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-09-17T23:22:02.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-09-17T23:22:02.699 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T23:22:02.709 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T23:22:02.712 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T23:22:02.819 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T23:22:02.830 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T23:22:02.831 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi037.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T23:25:50.440 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-09-17T23:25:50.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-17T23:25:50.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:25:50.519 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-17T23:25:50.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-17T23:25:50.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T23:25:50.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:25:50.685 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-17T23:25:50.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-17T23:25:50.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:25:50.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T23:25:50.841 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-17T23:25:50.925 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-17T23:25:50.926 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T23:25:50.937 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T23:25:50.937 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T23:25:50.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:25:50.940 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T23:25:50.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:25:50.943 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T23:25:50.943 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:25:50.971 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T23:25:50.975 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T23:25:50.988 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T23:25:50.992 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T23:25:50.995 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T23:25:51.010 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T23:25:51.015 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-17T23:25:51.020 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-09-17T23:25:51.027 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-17T23:25:51.032 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-09-17T23:25:51.035 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-09-17T23:25:51.042 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T23:25:51.046 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-09-17T23:25:51.048 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T23:25:51.057 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T23:25:51.060 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T23:25:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T23:25:51.074 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T23:25:51.115 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:25:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:25:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-17T23:25:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.118 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.122 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:25:51.127 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:25:51.127 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-17T23:25:51.127 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.127 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.127 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.127 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.129 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:25:51.132 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:25:51.132 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-17T23:25:51.132 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.132 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.132 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.132 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:25:51.133 INFO:teuthology.run_tasks:Running task install... 2024-09-17T23:25:51.142 DEBUG:teuthology.task.install:project ceph 2024-09-17T23:25:51.142 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-17T23:25:51.142 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-17T23:25:51.142 INFO:teuthology.task.install:Using flavor: default 2024-09-17T23:25:51.150 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T23:25:51.150 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-17T23:25:51.150 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T23:25:51.151 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:25:51.152 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T23:25:51.153 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:25:51.153 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T23:25:51.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:25:51.294 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T23:25:51.295 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T23:25:51.298 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T23:25:51.298 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T23:25:51.306 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T23:25:51.306 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T23:25:51.405 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T23:25:51.406 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:25:51.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T23:25:51.413 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T23:25:51.413 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:25:51.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T23:25:51.416 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T23:25:51.416 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:25:51.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T23:25:51.433 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-17T23:25:51.433 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T23:25:51.439 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-17T23:25:51.439 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T23:25:51.445 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-17T23:25:51.445 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T23:25:51.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T23:25:51.505 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T23:25:51.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T23:25:51.577 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T23:25:51.580 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T23:25:51.591 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T23:25:51.643 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-17T23:25:51.644 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-09-17T23:25:51.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-17T23:25:51.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-09-17T23:25:51.656 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-09-17T23:25:51.658 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-09-17T23:25:51.877 INFO:teuthology.orchestra.run.smithi139.stdout:45 files removed 2024-09-17T23:25:51.885 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-17T23:25:51.900 INFO:teuthology.orchestra.run.smithi037.stdout:45 files removed 2024-09-17T23:25:51.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw 2024-09-17T23:25:51.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-09-17T23:25:51.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw 2024-09-17T23:25:52.455 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-17T23:25:52.479 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-17T23:25:52.484 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 322 kB/s | 84 kB 00:00 2024-09-17T23:25:52.739 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-17T23:25:52.760 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-17T23:25:52.771 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-17T23:25:53.014 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T23:25:53.019 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-17T23:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-17T23:25:53.469 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-17T23:25:53.472 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-17T23:25:53.622 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-17T23:25:55.395 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-17T23:26:00.097 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 20 MB 00:05 2024-09-17T23:26:01.896 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-17T23:26:02.556 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-17T23:26:04.405 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-17T23:26:05.985 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-17T23:26:06.807 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-17T23:26:07.197 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.5 MB 00:03 2024-09-17T23:26:08.616 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-17T23:26:08.832 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-17T23:26:10.578 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 12 MB/s | 23 MB 00:01 2024-09-17T23:26:10.852 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 12 MB/s | 23 MB 00:01 2024-09-17T23:26:11.690 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-17T23:26:13.995 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T23:26:14.008 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T23:26:14.009 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T23:26:14.010 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout:Install 30 Packages 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 2 Packages 2024-09-17T23:26:14.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:14.012 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 60 M 2024-09-17T23:26:14.012 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:26:15.008 INFO:teuthology.orchestra.run.smithi037.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 9.6 MB/s | 5.4 MB 00:00 2024-09-17T23:26:15.058 INFO:teuthology.orchestra.run.smithi037.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 506 kB/s | 25 kB 00:00 2024-09-17T23:26:15.141 INFO:teuthology.orchestra.run.smithi037.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 8.8 MB/s | 751 kB 00:00 2024-09-17T23:26:15.275 INFO:teuthology.orchestra.run.smithi037.stdout:(4/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 13 MB/s | 11 MB 00:00 2024-09-17T23:26:15.317 INFO:teuthology.orchestra.run.smithi037.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 2.8 MB/s | 506 kB 00:00 2024-09-17T23:26:15.384 INFO:teuthology.orchestra.run.smithi037.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 677 kB/s | 45 kB 00:00 2024-09-17T23:26:15.451 INFO:teuthology.orchestra.run.smithi037.stdout:(7/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.0 MB/s | 136 kB 00:00 2024-09-17T23:26:15.793 INFO:teuthology.orchestra.run.smithi037.stdout:(8/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 16 MB/s | 21 MB 00:01 2024-09-17T23:26:15.826 INFO:teuthology.orchestra.run.smithi037.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 431 kB/s | 162 kB 00:00 2024-09-17T23:26:15.901 INFO:teuthology.orchestra.run.smithi037.stdout:(10/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 8.5 MB/s | 5.3 MB 00:00 2024-09-17T23:26:15.926 INFO:teuthology.orchestra.run.smithi037.stdout:(11/32): python3-rados-19.1.1-330.gc2ddf972.el9 2.3 MB/s | 320 kB 00:00 2024-09-17T23:26:15.952 INFO:teuthology.orchestra.run.smithi037.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T23:26:15.977 INFO:teuthology.orchestra.run.smithi037.stdout:(13/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 300 kB 00:00 2024-09-17T23:26:16.052 INFO:teuthology.orchestra.run.smithi037.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 853 kB/s | 107 kB 00:00 2024-09-17T23:26:16.077 INFO:teuthology.orchestra.run.smithi037.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-09-17T23:26:16.103 INFO:teuthology.orchestra.run.smithi037.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-17T23:26:16.109 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-17T23:26:16.145 INFO:teuthology.orchestra.run.smithi037.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-17T23:26:16.178 INFO:teuthology.orchestra.run.smithi037.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.1 MB/s | 662 kB 00:00 2024-09-17T23:26:16.204 INFO:teuthology.orchestra.run.smithi037.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-17T23:26:16.229 INFO:teuthology.orchestra.run.smithi037.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-09-17T23:26:16.254 INFO:teuthology.orchestra.run.smithi037.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-17T23:26:16.280 INFO:teuthology.orchestra.run.smithi037.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-17T23:26:16.347 INFO:teuthology.orchestra.run.smithi037.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-09-17T23:26:16.372 INFO:teuthology.orchestra.run.smithi037.stdout:(24/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-17T23:26:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-09-17T23:26:16.422 INFO:teuthology.orchestra.run.smithi037.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T23:26:16.472 INFO:teuthology.orchestra.run.smithi037.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-17T23:26:16.565 INFO:teuthology.orchestra.run.smithi037.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-09-17T23:26:16.599 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-17T23:26:16.765 INFO:teuthology.orchestra.run.smithi037.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 17 MB/s | 3.4 MB 00:00 2024-09-17T23:26:16.807 INFO:teuthology.orchestra.run.smithi037.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.7 MB/s | 1.6 MB 00:00 2024-09-17T23:26:17.032 INFO:teuthology.orchestra.run.smithi037.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-17T23:26:17.124 INFO:teuthology.orchestra.run.smithi037.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-09-17T23:26:17.128 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:17.128 INFO:teuthology.orchestra.run.smithi037.stdout:Total 19 MB/s | 60 MB 00:03 2024-09-17T23:26:17.239 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:26:17.280 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:26:17.280 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:26:17.975 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:26:17.975 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:26:18.320 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T23:26:18.327 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T23:26:18.328 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T23:26:18.329 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout:Install 30 Packages 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 2 Packages 2024-09-17T23:26:18.330 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:18.331 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 60 M 2024-09-17T23:26:18.331 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:26:18.895 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:26:18.907 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T23:26:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T23:26:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:26:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Install 30 Packages 2024-09-17T23:26:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-09-17T23:26:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 60 M 2024-09-17T23:26:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:26:18.960 INFO:teuthology.orchestra.run.smithi139.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 9.9 MB/s | 5.4 MB 00:00 2024-09-17T23:26:19.019 INFO:teuthology.orchestra.run.smithi139.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 432 kB/s | 25 kB 00:00 2024-09-17T23:26:19.127 INFO:teuthology.orchestra.run.smithi139.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 6.8 MB/s | 751 kB 00:00 2024-09-17T23:26:19.253 INFO:teuthology.orchestra.run.smithi139.stdout:(4/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 13 MB/s | 11 MB 00:00 2024-09-17T23:26:19.278 INFO:teuthology.orchestra.run.smithi139.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 3.3 MB/s | 506 kB 00:00 2024-09-17T23:26:19.489 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:26:19.569 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T23:26:19.603 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T23:26:19.637 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T23:26:19.651 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T23:26:19.680 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T23:26:19.745 INFO:teuthology.orchestra.run.smithi139.stdout:(6/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 11 MB/s | 5.3 MB 00:00 2024-09-17T23:26:19.820 INFO:teuthology.orchestra.run.smithi139.stdout:(7/32): python3-ceph-argparse-19.1.1-330.gc2ddf 83 kB/s | 45 kB 00:00 2024-09-17T23:26:19.845 INFO:teuthology.orchestra.run.smithi139.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 1.3 MB/s | 136 kB 00:00 2024-09-17T23:26:19.896 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T23:26:19.904 INFO:teuthology.orchestra.run.smithi139.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 1.9 MB/s | 162 kB 00:00 2024-09-17T23:26:19.929 INFO:teuthology.orchestra.run.smithi139.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 3.7 MB/s | 320 kB 00:00 2024-09-17T23:26:19.950 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T23:26:19.959 INFO:teuthology.orchestra.run.smithi031.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 5.6 MB/s | 5.4 MB 00:00 2024-09-17T23:26:20.017 INFO:teuthology.orchestra.run.smithi031.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 433 kB/s | 25 kB 00:00 2024-09-17T23:26:20.130 INFO:teuthology.orchestra.run.smithi139.stdout:(11/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 12 MB/s | 21 MB 00:01 2024-09-17T23:26:20.155 INFO:teuthology.orchestra.run.smithi139.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 441 kB/s | 99 kB 00:00 2024-09-17T23:26:20.180 INFO:teuthology.orchestra.run.smithi139.stdout:(13/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 1.1 MB/s | 300 kB 00:00 2024-09-17T23:26:20.209 INFO:teuthology.orchestra.run.smithi031.stdout:(3/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 8.7 MB/s | 11 MB 00:01 2024-09-17T23:26:20.322 INFO:teuthology.orchestra.run.smithi139.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 557 kB/s | 107 kB 00:00 2024-09-17T23:26:20.347 INFO:teuthology.orchestra.run.smithi139.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-09-17T23:26:20.355 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T23:26:20.372 INFO:teuthology.orchestra.run.smithi139.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 737 kB/s | 160 kB 00:00 2024-09-17T23:26:20.413 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:(4/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 15 MB/s | 21 MB 00:01 2024-09-17T23:26:20.423 INFO:teuthology.orchestra.run.smithi139.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-17T23:26:20.460 INFO:teuthology.orchestra.run.smithi031.stdout:(5/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 1.7 MB/s | 751 kB 00:00 2024-09-17T23:26:20.481 INFO:teuthology.orchestra.run.smithi139.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-17T23:26:20.493 INFO:teuthology.orchestra.run.smithi031.stdout:(6/32): libradosstriper1-19.1.1-330.gc2ddf972.e 1.7 MB/s | 506 kB 00:00 2024-09-17T23:26:20.506 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:20.515 INFO:teuthology.orchestra.run.smithi139.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-17T23:26:20.519 INFO:teuthology.orchestra.run.smithi031.stdout:(7/32): python3-ceph-argparse-19.1.1-330.gc2ddf 771 kB/s | 45 kB 00:00 2024-09-17T23:26:20.540 INFO:teuthology.orchestra.run.smithi139.stdout:(20/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-17T23:26:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.3 MB/s | 136 kB 00:00 2024-09-17T23:26:20.566 INFO:teuthology.orchestra.run.smithi139.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 426 kB/s | 36 kB 00:00 2024-09-17T23:26:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 2.4 MB/s | 162 kB 00:00 2024-09-17T23:26:20.591 INFO:teuthology.orchestra.run.smithi139.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 625 kB/s | 47 kB 00:00 2024-09-17T23:26:20.596 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T23:26:20.611 INFO:teuthology.orchestra.run.smithi031.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 5.3 MB/s | 320 kB 00:00 2024-09-17T23:26:20.623 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T23:26:20.636 INFO:teuthology.orchestra.run.smithi031.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-17T23:26:20.658 INFO:teuthology.orchestra.run.smithi139.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 500 kB/s | 26 kB 00:00 2024-09-17T23:26:20.661 INFO:teuthology.orchestra.run.smithi031.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T23:26:20.677 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T23:26:20.700 INFO:teuthology.orchestra.run.smithi139.stdout:(24/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.9 MB/s | 308 kB 00:00 2024-09-17T23:26:20.706 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T23:26:20.725 INFO:teuthology.orchestra.run.smithi139.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 725 kB/s | 49 kB 00:00 2024-09-17T23:26:20.745 INFO:teuthology.orchestra.run.smithi031.stdout:(13/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 16 MB/s | 5.3 MB 00:00 2024-09-17T23:26:20.751 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T23:26:20.775 INFO:teuthology.orchestra.run.smithi139.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-09-17T23:26:20.787 INFO:teuthology.orchestra.run.smithi031.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 710 kB/s | 107 kB 00:00 2024-09-17T23:26:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T23:26:20.837 INFO:teuthology.orchestra.run.smithi031.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-09-17T23:26:20.842 INFO:teuthology.orchestra.run.smithi139.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-17T23:26:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:20.871 INFO:teuthology.orchestra.run.smithi031.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 766 kB/s | 160 kB 00:00 2024-09-17T23:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-17T23:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 842 kB/s | 42 kB 00:00 2024-09-17T23:26:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:20.934 INFO:teuthology.orchestra.run.smithi139.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-17T23:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T23:26:20.957 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T23:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-17T23:26:20.997 INFO:teuthology.orchestra.run.smithi031.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 475 kB/s | 36 kB 00:00 2024-09-17T23:26:21.023 INFO:teuthology.orchestra.run.smithi031.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-17T23:26:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T23:26:21.089 INFO:teuthology.orchestra.run.smithi031.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 508 kB/s | 26 kB 00:00 2024-09-17T23:26:21.100 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T23:26:21.110 INFO:teuthology.orchestra.run.smithi139.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 20 MB/s | 3.4 MB 00:00 2024-09-17T23:26:21.123 INFO:teuthology.orchestra.run.smithi031.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-17T23:26:21.148 INFO:teuthology.orchestra.run.smithi031.stdout:(25/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.7 MB/s | 308 kB 00:00 2024-09-17T23:26:21.148 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T23:26:21.168 INFO:teuthology.orchestra.run.smithi139.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-17T23:26:21.174 INFO:teuthology.orchestra.run.smithi031.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T23:26:21.181 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T23:26:21.210 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T23:26:21.224 INFO:teuthology.orchestra.run.smithi031.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-17T23:26:21.244 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T23:26:21.260 INFO:teuthology.orchestra.run.smithi139.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 21 MB/s | 3.2 MB 00:00 2024-09-17T23:26:21.291 INFO:teuthology.orchestra.run.smithi031.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 839 kB 00:00 2024-09-17T23:26:21.452 INFO:teuthology.orchestra.run.smithi139.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2024-09-17T23:26:21.457 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:21.457 INFO:teuthology.orchestra.run.smithi139.stdout:Total 19 MB/s | 60 MB 00:03 2024-09-17T23:26:21.466 INFO:teuthology.orchestra.run.smithi031.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-09-17T23:26:21.550 INFO:teuthology.orchestra.run.smithi031.stdout:(30/32): librados2-19.1.1-330.gc2ddf972.el9.x86 13 MB/s | 3.4 MB 00:00 2024-09-17T23:26:21.562 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:26:21.575 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T23:26:21.598 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:26:21.598 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:26:21.614 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T23:26:21.675 INFO:teuthology.orchestra.run.smithi031.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 15 MB/s | 3.2 MB 00:00 2024-09-17T23:26:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T23:26:21.784 INFO:teuthology.orchestra.run.smithi031.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-09-17T23:26:21.789 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:21.789 INFO:teuthology.orchestra.run.smithi031.stdout:Total 21 MB/s | 60 MB 00:02 2024-09-17T23:26:21.803 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T23:26:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T23:26:21.898 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:26:21.939 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:26:21.939 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:26:22.290 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:26:22.291 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:26:22.360 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T23:26:22.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T23:26:22.455 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T23:26:22.527 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T23:26:22.619 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:26:22.619 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:26:22.677 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T23:26:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:26:23.842 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T23:26:23.873 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T23:26:23.899 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T23:26:23.921 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T23:26:23.945 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T23:26:24.123 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:26:24.171 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T23:26:24.204 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T23:26:24.226 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T23:26:24.238 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T23:26:24.264 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T23:26:24.286 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T23:26:24.309 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T23:26:24.506 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T23:26:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T23:26:24.565 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T23:26:24.675 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T23:26:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T23:26:24.932 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T23:26:24.960 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T23:26:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T23:26:25.012 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T23:26:25.032 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:25.075 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:25.090 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T23:26:25.112 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:25.115 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:25.176 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:25.210 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T23:26:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T23:26:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T23:26:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T23:26:25.329 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T23:26:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T23:26:25.358 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T23:26:25.403 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T23:26:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T23:26:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T23:26:25.464 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T23:26:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T23:26:25.497 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T23:26:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T23:26:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T23:26:25.714 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T23:26:25.760 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T23:26:25.792 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T23:26:25.808 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T23:26:25.825 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T23:26:25.843 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T23:26:25.851 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T23:26:25.852 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T23:26:25.852 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T23:26:25.852 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:25.854 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T23:26:25.886 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:25.887 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T23:26:25.934 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T23:26:25.973 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T23:26:26.017 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T23:26:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T23:26:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T23:26:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T23:26:26.388 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T23:26:26.424 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T23:26:26.499 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T23:26:26.528 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T23:26:26.583 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T23:26:26.660 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T23:26:26.689 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T23:26:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T23:26:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T23:26:27.040 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T23:26:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T23:26:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T23:26:29.001 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:29.052 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:29.526 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:29.567 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:29.663 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T23:26:29.770 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T23:26:29.770 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T23:26:29.771 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T23:26:29.771 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:29.805 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T23:26:30.195 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T23:26:30.289 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T23:26:30.290 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T23:26:30.290 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T23:26:30.290 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:30.329 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-09-17T23:26:36.401 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:37.426 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T23:26:37.453 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T23:26:37.453 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:26:37.453 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T23:26:37.453 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T23:26:37.454 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T23:26:37.454 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:37.456 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T23:26:37.545 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T23:26:37.546 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T23:26:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T23:26:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T23:26:38.747 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T23:26:38.748 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:38.749 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T23:26:38.750 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T23:26:39.805 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:39.805 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-09-17T23:26:39.805 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-09-17T23:26:39.805 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-09-17T23:26:39.805 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-09-17T23:26:39.806 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-09-17T23:26:39.806 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-09-17T23:26:39.806 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-09-17T23:26:39.806 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:39.879 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:39.879 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:39.879 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-09-17T23:26:39.879 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T23:26:39.880 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:39.882 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:26:40.247 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-test 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-17T23:26:40.657 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-17T23:26:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:40.821 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T23:26:40.847 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T23:26:40.847 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:26:40.847 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T23:26:40.847 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T23:26:40.847 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T23:26:40.847 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:40.849 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T23:26:40.850 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:26:40.940 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T23:26:40.940 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:40.998 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T23:26:40.999 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout:Install 4 Packages 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 49 M 2024-09-17T23:26:41.000 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 209 M 2024-09-17T23:26:41.001 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:26:41.217 INFO:teuthology.orchestra.run.smithi037.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-17T23:26:41.242 INFO:teuthology.orchestra.run.smithi037.stdout:(2/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 752 kB/s | 163 kB 00:00 2024-09-17T23:26:41.267 INFO:teuthology.orchestra.run.smithi037.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-17T23:26:41.669 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T23:26:41.696 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T23:26:41.696 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:26:41.696 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T23:26:41.697 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T23:26:41.697 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T23:26:41.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:41.698 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T23:26:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T23:26:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:42.169 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:42.169 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T23:26:42.169 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T23:26:42.169 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T23:26:42.169 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T23:26:42.170 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T23:26:42.171 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T23:26:42.172 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:42.172 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T23:26:42.172 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T23:26:42.609 INFO:teuthology.orchestra.run.smithi037.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 31 MB/s | 49 MB 00:01 2024-09-17T23:26:42.611 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:42.611 INFO:teuthology.orchestra.run.smithi037.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-17T23:26:42.640 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:26:42.662 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:26:42.662 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:26:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T23:26:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T23:26:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T23:26:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T23:26:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T23:26:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T23:26:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T23:26:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T23:26:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T23:26:43.067 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:26:43.068 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:26:43.391 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T23:26:43.392 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.393 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:43.394 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:26:43.624 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:26:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T23:26:43.759 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T23:26:43.786 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T23:26:43.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-test 2024-09-17T23:26:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T23:26:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:44.238 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-17T23:26:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T23:26:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:44.241 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:26:44.398 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:26:44.539 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:44.540 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:26:44.541 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:44.541 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-17T23:26:44.541 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:44.541 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 49 M 2024-09-17T23:26:44.541 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 209 M 2024-09-17T23:26:44.541 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:26:44.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-09-17T23:26:44.745 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-17T23:26:44.787 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T23:26:44.812 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 731 kB/s | 163 kB 00:00 2024-09-17T23:26:45.196 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:26:45.335 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:26:45.335 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:45.335 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-17T23:26:45.336 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:45.337 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 49 M 2024-09-17T23:26:45.337 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 209 M 2024-09-17T23:26:45.337 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:26:45.618 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-17T23:26:45.668 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-09-17T23:26:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 443 kB/s | 163 kB 00:00 2024-09-17T23:26:46.087 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-17T23:26:46.089 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:46.090 INFO:teuthology.orchestra.run.smithi139.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-17T23:26:46.117 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:26:46.137 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:26:46.137 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:26:46.519 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:26:46.519 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:26:47.018 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 29 MB/s | 49 MB 00:01 2024-09-17T23:26:47.020 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:47.021 INFO:teuthology.orchestra.run.smithi031.stdout:Total 29 MB/s | 49 MB 00:01 2024-09-17T23:26:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:26:47.050 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:26:47.072 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:26:47.072 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:26:47.129 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T23:26:47.170 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T23:26:47.206 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T23:26:47.481 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:26:47.481 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:26:48.041 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:26:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T23:26:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T23:26:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T23:26:48.696 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T23:26:48.787 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T23:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T23:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T23:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T23:26:51.125 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T23:26:51.449 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:51.450 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:26:51.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph 2024-09-17T23:26:51.831 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T23:26:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T23:26:52.334 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:26:52.476 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:26:52.478 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:52.478 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T23:26:52.479 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T23:26:52.480 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T23:26:52.481 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout:Install 37 Packages 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 30 M 2024-09-17T23:26:52.482 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 108 M 2024-09-17T23:26:52.483 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:26:53.142 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T23:26:53.230 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T23:26:53.485 INFO:teuthology.orchestra.run.smithi037.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 44 kB/s | 6.5 kB 00:00 2024-09-17T23:26:53.660 INFO:teuthology.orchestra.run.smithi037.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-17T23:26:53.710 INFO:teuthology.orchestra.run.smithi037.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.3 MB/s | 2.4 MB 00:00 2024-09-17T23:26:53.769 INFO:teuthology.orchestra.run.smithi037.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 4.2 MB/s | 249 kB 00:00 2024-09-17T23:26:54.236 INFO:teuthology.orchestra.run.smithi037.stdout:(5/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-17T23:26:54.303 INFO:teuthology.orchestra.run.smithi037.stdout:(6/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 5.8 MB/s | 4.7 MB 00:00 2024-09-17T23:26:54.420 INFO:teuthology.orchestra.run.smithi037.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 389 kB/s | 253 kB 00:00 2024-09-17T23:26:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T23:26:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T23:26:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T23:26:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T23:26:54.537 INFO:teuthology.orchestra.run.smithi037.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-17T23:26:54.654 INFO:teuthology.orchestra.run.smithi037.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.0 MB/s | 125 kB 00:00 2024-09-17T23:26:54.762 INFO:teuthology.orchestra.run.smithi037.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 232 kB/s | 106 kB 00:00 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:54.779 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:26:54.788 INFO:teuthology.orchestra.run.smithi037.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.6 MB/s | 215 kB 00:00 2024-09-17T23:26:54.896 INFO:teuthology.orchestra.run.smithi037.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-17T23:26:54.925 INFO:teuthology.orchestra.run.smithi037.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-17T23:26:54.941 INFO:teuthology.orchestra.run.smithi037.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 774 kB/s | 35 kB 00:00 2024-09-17T23:26:54.967 INFO:teuthology.orchestra.run.smithi037.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-17T23:26:54.992 INFO:teuthology.orchestra.run.smithi037.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T23:26:55.017 INFO:teuthology.orchestra.run.smithi037.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-17T23:26:55.087 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph 2024-09-17T23:26:55.092 INFO:teuthology.orchestra.run.smithi037.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-17T23:26:55.126 INFO:teuthology.orchestra.run.smithi037.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-09-17T23:26:55.152 INFO:teuthology.orchestra.run.smithi037.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-09-17T23:26:55.185 INFO:teuthology.orchestra.run.smithi037.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-17T23:26:55.210 INFO:teuthology.orchestra.run.smithi037.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 465 kB/s | 11 kB 00:00 2024-09-17T23:26:55.236 INFO:teuthology.orchestra.run.smithi037.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 708 kB/s | 18 kB 00:00 2024-09-17T23:26:55.269 INFO:teuthology.orchestra.run.smithi037.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.0 MB/s | 358 kB 00:00 2024-09-17T23:26:55.295 INFO:teuthology.orchestra.run.smithi037.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-09-17T23:26:55.320 INFO:teuthology.orchestra.run.smithi037.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-17T23:26:55.345 INFO:teuthology.orchestra.run.smithi037.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-17T23:26:55.370 INFO:teuthology.orchestra.run.smithi037.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-17T23:26:55.396 INFO:teuthology.orchestra.run.smithi037.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T23:26:55.421 INFO:teuthology.orchestra.run.smithi037.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-09-17T23:26:55.446 INFO:teuthology.orchestra.run.smithi037.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-17T23:26:55.472 INFO:teuthology.orchestra.run.smithi037.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T23:26:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-17T23:26:55.539 INFO:teuthology.orchestra.run.smithi037.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-17T23:26:55.572 INFO:teuthology.orchestra.run.smithi037.stdout:(35/37): python3-cryptography-36.0.1-4.el9.x86_ 940 kB/s | 1.2 MB 00:01 2024-09-17T23:26:55.598 INFO:teuthology.orchestra.run.smithi037.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-17T23:26:55.623 INFO:teuthology.orchestra.run.smithi037.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-17T23:26:55.626 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:55.626 INFO:teuthology.orchestra.run.smithi037.stdout:Total 9.6 MB/s | 30 MB 00:03 2024-09-17T23:26:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T23:26:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T23:26:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T23:26:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T23:26:55.681 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:26:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:26:55.823 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:26:55.826 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:55.826 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:26:55.826 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:55.826 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T23:26:55.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T23:26:55.828 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T23:26:55.829 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T23:26:55.830 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T23:26:55.830 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:55.830 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:26:55.830 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:26:55.830 INFO:teuthology.orchestra.run.smithi139.stdout:Install 37 Packages 2024-09-17T23:26:55.830 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:26:55.831 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:26:55.831 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:26:55.831 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 30 M 2024-09-17T23:26:55.831 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 108 M 2024-09-17T23:26:55.831 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T23:26:56.033 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T23:26:56.034 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:56.034 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:26:56.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-09-17T23:26:56.348 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:26:56.348 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:26:56.392 INFO:teuthology.orchestra.run.smithi139.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 48 kB/s | 6.5 kB 00:00 2024-09-17T23:26:56.609 INFO:teuthology.orchestra.run.smithi139.stdout:(2/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.8 MB/s | 2.4 MB 00:00 2024-09-17T23:26:56.651 INFO:teuthology.orchestra.run.smithi139.stdout:(3/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 3.8 MB/s | 1.5 MB 00:00 2024-09-17T23:26:56.785 INFO:teuthology.orchestra.run.smithi139.stdout:(4/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-17T23:26:56.812 INFO:teuthology.orchestra.run.smithi139.stdout:(5/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 1.5 MB/s | 249 kB 00:00 2024-09-17T23:26:56.925 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:40 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:26:56.960 INFO:teuthology.orchestra.run.smithi139.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-17T23:26:57.010 INFO:teuthology.orchestra.run.smithi139.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-17T23:26:57.068 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:26:57.071 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:57.071 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:26:57.071 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:57.071 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:26:57.071 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T23:26:57.071 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T23:26:57.072 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T23:26:57.073 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:26:57.074 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:26:57.075 INFO:teuthology.orchestra.run.smithi031.stdout:Install 37 Packages 2024-09-17T23:26:57.075 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:26:57.075 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 30 M 2024-09-17T23:26:57.075 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 108 M 2024-09-17T23:26:57.075 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:26:57.169 INFO:teuthology.orchestra.run.smithi139.stdout:(8/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-17T23:26:57.194 INFO:teuthology.orchestra.run.smithi139.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 736 kB/s | 135 kB 00:00 2024-09-17T23:26:57.236 INFO:teuthology.orchestra.run.smithi139.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-17T23:26:57.261 INFO:teuthology.orchestra.run.smithi139.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-17T23:26:57.328 INFO:teuthology.orchestra.run.smithi139.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 786 kB/s | 125 kB 00:00 2024-09-17T23:26:57.387 INFO:teuthology.orchestra.run.smithi139.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-09-17T23:26:57.412 INFO:teuthology.orchestra.run.smithi139.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 415 kB/s | 35 kB 00:00 2024-09-17T23:26:57.437 INFO:teuthology.orchestra.run.smithi139.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 979 kB/s | 172 kB 00:00 2024-09-17T23:26:57.463 INFO:teuthology.orchestra.run.smithi139.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-17T23:26:57.488 INFO:teuthology.orchestra.run.smithi139.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-17T23:26:57.513 INFO:teuthology.orchestra.run.smithi139.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 550 kB/s | 42 kB 00:00 2024-09-17T23:26:57.581 INFO:teuthology.orchestra.run.smithi139.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-17T23:26:57.606 INFO:teuthology.orchestra.run.smithi139.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 156 kB/s | 22 kB 00:00 2024-09-17T23:26:57.631 INFO:teuthology.orchestra.run.smithi139.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-09-17T23:26:57.664 INFO:teuthology.orchestra.run.smithi139.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 415 kB/s | 11 kB 00:00 2024-09-17T23:26:57.690 INFO:teuthology.orchestra.run.smithi139.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-17T23:26:57.715 INFO:teuthology.orchestra.run.smithi139.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 363 kB/s | 18 kB 00:00 2024-09-17T23:26:57.740 INFO:teuthology.orchestra.run.smithi139.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-09-17T23:26:57.765 INFO:teuthology.orchestra.run.smithi139.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-17T23:26:57.799 INFO:teuthology.orchestra.run.smithi139.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-09-17T23:26:57.824 INFO:teuthology.orchestra.run.smithi139.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 792 kB/s | 46 kB 00:00 2024-09-17T23:26:57.848 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:26:57.849 INFO:teuthology.orchestra.run.smithi139.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T23:26:57.875 INFO:teuthology.orchestra.run.smithi139.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-17T23:26:57.900 INFO:teuthology.orchestra.run.smithi139.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-17T23:26:57.927 INFO:teuthology.orchestra.run.smithi031.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 26 kB/s | 6.5 kB 00:00 2024-09-17T23:26:57.934 INFO:teuthology.orchestra.run.smithi139.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-17T23:26:57.959 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T23:26:57.959 INFO:teuthology.orchestra.run.smithi139.stdout:(33/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-17T23:26:57.984 INFO:teuthology.orchestra.run.smithi139.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 427 kB/s | 36 kB 00:00 2024-09-17T23:26:57.992 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T23:26:58.018 INFO:teuthology.orchestra.run.smithi139.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 608 kB/s | 20 kB 00:00 2024-09-17T23:26:58.019 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T23:26:58.051 INFO:teuthology.orchestra.run.smithi139.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-17T23:26:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T23:26:58.110 INFO:teuthology.orchestra.run.smithi139.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-17T23:26:58.114 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:26:58.114 INFO:teuthology.orchestra.run.smithi139.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-17T23:26:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T23:26:58.265 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:26:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T23:26:58.305 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:26:58.305 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:26:58.389 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T23:26:58.474 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T23:26:58.521 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T23:26:58.556 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T23:26:58.586 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T23:26:58.620 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T23:26:58.654 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T23:26:58.695 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T23:26:58.729 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T23:26:58.763 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T23:26:58.778 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:26:58.779 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:26:58.813 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T23:26:58.852 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T23:26:58.918 INFO:teuthology.orchestra.run.smithi031.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 1.2 MB/s | 1.5 MB 00:01 2024-09-17T23:26:58.936 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T23:26:59.087 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T23:26:59.194 INFO:teuthology.orchestra.run.smithi031.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 1.6 MB/s | 2.4 MB 00:01 2024-09-17T23:26:59.254 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T23:26:59.261 INFO:teuthology.orchestra.run.smithi031.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 3.7 MB/s | 249 kB 00:00 2024-09-17T23:26:59.305 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T23:26:59.343 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T23:26:59.401 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T23:26:59.428 INFO:teuthology.orchestra.run.smithi031.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-17T23:26:59.465 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T23:26:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:(6/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 2.8 MB/s | 4.7 MB 00:01 2024-09-17T23:26:59.659 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T23:26:59.705 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T23:26:59.753 INFO:teuthology.orchestra.run.smithi031.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 3.8 MB/s | 1.2 MB 00:00 2024-09-17T23:26:59.782 INFO:teuthology.orchestra.run.smithi031.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 546 kB/s | 106 kB 00:00 2024-09-17T23:26:59.807 INFO:teuthology.orchestra.run.smithi031.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.5 MB/s | 135 kB 00:00 2024-09-17T23:26:59.810 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T23:26:59.857 INFO:teuthology.orchestra.run.smithi031.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-09-17T23:26:59.891 INFO:teuthology.orchestra.run.smithi031.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-17T23:26:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T23:26:59.941 INFO:teuthology.orchestra.run.smithi031.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-17T23:26:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T23:26:59.991 INFO:teuthology.orchestra.run.smithi031.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-17T23:27:00.017 INFO:teuthology.orchestra.run.smithi031.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 461 kB/s | 35 kB 00:00 2024-09-17T23:27:00.042 INFO:teuthology.orchestra.run.smithi031.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-17T23:27:00.067 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T23:27:00.067 INFO:teuthology.orchestra.run.smithi031.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T23:27:00.093 INFO:teuthology.orchestra.run.smithi031.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-09-17T23:27:00.168 INFO:teuthology.orchestra.run.smithi031.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-17T23:27:00.170 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:00.233 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T23:27:00.237 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T23:27:00.264 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T23:27:00.281 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T23:27:00.291 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T23:27:00.291 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:00.292 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T23:27:00.292 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T23:27:00.292 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T23:27:00.292 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:00.315 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T23:27:00.351 INFO:teuthology.orchestra.run.smithi031.stdout:(19/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_ 12 MB/s | 17 MB 00:01 2024-09-17T23:27:00.377 INFO:teuthology.orchestra.run.smithi031.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 208 kB/s | 43 kB 00:00 2024-09-17T23:27:00.385 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T23:27:00.448 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T23:27:00.469 INFO:teuthology.orchestra.run.smithi031.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T23:27:00.480 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T23:27:00.502 INFO:teuthology.orchestra.run.smithi031.stdout:(22/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.0 MB/s | 358 kB 00:00 2024-09-17T23:27:00.527 INFO:teuthology.orchestra.run.smithi031.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-09-17T23:27:00.553 INFO:teuthology.orchestra.run.smithi031.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-17T23:27:00.578 INFO:teuthology.orchestra.run.smithi031.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-17T23:27:00.579 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T23:27:00.603 INFO:teuthology.orchestra.run.smithi031.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 395 kB/s | 19 kB 00:00 2024-09-17T23:27:00.628 INFO:teuthology.orchestra.run.smithi031.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-17T23:27:00.654 INFO:teuthology.orchestra.run.smithi031.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-17T23:27:00.666 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T23:27:00.687 INFO:teuthology.orchestra.run.smithi031.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-17T23:27:00.707 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T23:27:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 493 kB/s | 16 kB 00:00 2024-09-17T23:27:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T23:27:00.746 INFO:teuthology.orchestra.run.smithi031.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-17T23:27:00.772 INFO:teuthology.orchestra.run.smithi031.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-17T23:27:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T23:27:00.797 INFO:teuthology.orchestra.run.smithi031.stdout:(33/37): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-09-17T23:27:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T23:27:00.822 INFO:teuthology.orchestra.run.smithi031.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 473 kB/s | 36 kB 00:00 2024-09-17T23:27:00.839 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T23:27:00.864 INFO:teuthology.orchestra.run.smithi031.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-17T23:27:00.881 INFO:teuthology.orchestra.run.smithi031.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-17T23:27:00.888 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T23:27:00.932 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T23:27:00.964 INFO:teuthology.orchestra.run.smithi031.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-09-17T23:27:00.966 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T23:27:00.970 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:00.970 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.8 MB/s | 30 MB 00:03 2024-09-17T23:27:01.014 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T23:27:01.046 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T23:27:01.119 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T23:27:01.123 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:01.168 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:01.168 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T23:27:01.338 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T23:27:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T23:27:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T23:27:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T23:27:01.542 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T23:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T23:27:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T23:27:01.823 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T23:27:01.854 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T23:27:01.854 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:01.854 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T23:27:01.854 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T23:27:01.854 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T23:27:01.854 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:01.877 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T23:27:01.974 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T23:27:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T23:27:02.133 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T23:27:02.287 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T23:27:02.315 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T23:27:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T23:27:02.341 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T23:27:02.341 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:02.341 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T23:27:02.341 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T23:27:02.341 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T23:27:02.341 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T23:27:02.344 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:02.344 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T23:27:02.345 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T23:27:02.345 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T23:27:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T23:27:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T23:27:02.608 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:02.608 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T23:27:02.608 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T23:27:02.608 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T23:27:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T23:27:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:03.251 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T23:27:03.292 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T23:27:03.320 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T23:27:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T23:27:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T23:27:03.469 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T23:27:03.566 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T23:27:03.649 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T23:27:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T23:27:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T23:27:03.770 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T23:27:03.785 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T23:27:03.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T23:27:03.812 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T23:27:03.812 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:03.812 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T23:27:03.813 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T23:27:03.813 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T23:27:03.813 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:03.838 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T23:27:03.879 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T23:27:03.922 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T23:27:03.965 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T23:27:04.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T23:27:04.054 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T23:27:04.130 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T23:27:04.253 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T23:27:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T23:27:04.277 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:04.277 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T23:27:04.277 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T23:27:04.277 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T23:27:04.277 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:04.408 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T23:27:04.443 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T23:27:04.444 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T23:27:04.445 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T23:27:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T23:27:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T23:27:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T23:27:04.446 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T23:27:04.498 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T23:27:04.512 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T23:27:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T23:27:04.535 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:04.535 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T23:27:04.535 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T23:27:04.535 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T23:27:04.535 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T23:27:04.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T23:27:04.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T23:27:04.770 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T23:27:04.834 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T23:27:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T23:27:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T23:27:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T23:27:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T23:27:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T23:27:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:05.539 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T23:27:05.540 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T23:27:05.541 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:05.542 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T23:27:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T23:27:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T23:27:05.753 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:05.753 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T23:27:05.753 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T23:27:05.753 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T23:27:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:05.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-base 2024-09-17T23:27:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T23:27:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T23:27:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T23:27:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T23:27:06.503 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:06.595 INFO:teuthology.orchestra.run.smithi037.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:06.639 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:06.640 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:06.640 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:06.708 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install cephadm 2024-09-17T23:27:07.173 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T23:27:07.173 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:07.173 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:07.173 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:07.173 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T23:27:07.174 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T23:27:07.175 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:07.176 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T23:27:07.308 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T23:27:07.313 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:07.313 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T23:27:07.313 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T23:27:07.313 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T23:27:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:07.444 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:07.445 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:07.445 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:07.445 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:07.445 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 765 k 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 771 k 2024-09-17T23:27:07.446 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:07.595 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-base 2024-09-17T23:27:07.758 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.4 MB/s | 765 kB 00:00 2024-09-17T23:27:07.759 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:07.759 INFO:teuthology.orchestra.run.smithi037.stdout:Total 2.4 MB/s | 765 kB 00:00 2024-09-17T23:27:07.759 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:07.765 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:07.765 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:07.772 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:07.772 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T23:27:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T23:27:07.802 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:07.802 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T23:27:07.802 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T23:27:07.802 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T23:27:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:07.849 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T23:27:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T23:27:08.069 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:08.069 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T23:27:08.069 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T23:27:08.069 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T23:27:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:08.135 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T23:27:08.181 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:08.271 INFO:teuthology.orchestra.run.smithi139.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:08.313 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:08.314 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:08.314 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:08.338 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:08.384 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install cephadm 2024-09-17T23:27:08.431 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:08.815 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:08.965 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:09.076 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:09.077 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:09.097 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:09.097 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 765 k 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 771 k 2024-09-17T23:27:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:09.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T23:27:09.378 INFO:teuthology.orchestra.run.smithi139.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-17T23:27:09.378 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:09.379 INFO:teuthology.orchestra.run.smithi139.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-17T23:27:09.379 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:09.384 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:09.384 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:09.390 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:09.390 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:09.462 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:09.888 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:09.931 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T23:27:09.947 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T23:27:09.948 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T23:27:09.949 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T23:27:10.025 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:10.029 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:10.029 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:10.029 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-09-17T23:27:10.029 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:10.030 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:10.031 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 145 k 2024-09-17T23:27:10.031 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 435 k 2024-09-17T23:27:10.031 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:10.322 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 500 kB/s | 145 kB 00:00 2024-09-17T23:27:10.323 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:10.323 INFO:teuthology.orchestra.run.smithi037.stdout:Total 497 kB/s | 145 kB 00:00 2024-09-17T23:27:10.323 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:10.331 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:10.331 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:10.392 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:10.393 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:10.425 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:10.552 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:10.641 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:10.641 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T23:27:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:10.695 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:10.696 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:10.696 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:10.696 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:10.696 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:10.696 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:10.918 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T23:27:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T23:27:10.962 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:10.962 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T23:27:10.963 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T23:27:10.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T23:27:10.965 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:10.966 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:11.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-09-17T23:27:11.540 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:11.628 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:11.628 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:11.628 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:11.628 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:11.628 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:11.628 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:11.676 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:11.676 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:11.676 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-17T23:27:11.676 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:11.676 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 145 k 2024-09-17T23:27:11.677 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 435 k 2024-09-17T23:27:11.678 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:11.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr 2024-09-17T23:27:11.869 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 759 kB/s | 145 kB 00:00 2024-09-17T23:27:11.870 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:11.870 INFO:teuthology.orchestra.run.smithi139.stdout:Total 753 kB/s | 145 kB 00:00 2024-09-17T23:27:11.870 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:11.877 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:11.877 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:11.895 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:11.934 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:11.934 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:11.987 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:12.032 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:12.033 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:27:12.033 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:12.098 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:12.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-09-17T23:27:12.187 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:12.212 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:12.212 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:12.212 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T23:27:12.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:12.479 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:12.570 INFO:teuthology.orchestra.run.smithi037.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:12.614 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:12.615 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:12.616 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:12.679 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T23:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:12.850 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:12.851 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:12.851 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:12.851 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 765 k 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 771 k 2024-09-17T23:27:12.852 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:13.125 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-17T23:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-17T23:27:13.126 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:13.131 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:13.138 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:13.167 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:13.167 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:13.167 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:13.168 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:13.168 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:13.168 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:13.276 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:13.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr 2024-09-17T23:27:13.414 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T23:27:13.415 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout:Install 12 Packages 2024-09-17T23:27:13.416 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:13.417 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 8.7 M 2024-09-17T23:27:13.417 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 104 M 2024-09-17T23:27:13.418 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:13.687 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:13.700 INFO:teuthology.orchestra.run.smithi037.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 213 kB/s | 32 kB 00:00 2024-09-17T23:27:13.725 INFO:teuthology.orchestra.run.smithi037.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 94 kB/s | 16 kB 00:00 2024-09-17T23:27:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:13.892 INFO:teuthology.orchestra.run.smithi037.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-09-17T23:27:13.959 INFO:teuthology.orchestra.run.smithi037.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 8.7 MB/s | 3.6 MB 00:00 2024-09-17T23:27:13.992 INFO:teuthology.orchestra.run.smithi037.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-09-17T23:27:14.009 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:14.051 INFO:teuthology.orchestra.run.smithi037.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 385 kB/s | 19 kB 00:00 2024-09-17T23:27:14.076 INFO:teuthology.orchestra.run.smithi037.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-17T23:27:14.100 INFO:teuthology.orchestra.run.smithi139.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:14.110 INFO:teuthology.orchestra.run.smithi037.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.3 MB/s | 144 kB 00:00 2024-09-17T23:27:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:14.142 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:14.143 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:14.143 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:14.143 INFO:teuthology.orchestra.run.smithi037.stdout:(9/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.4 MB/s | 863 kB 00:00 2024-09-17T23:27:14.169 INFO:teuthology.orchestra.run.smithi037.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 524 kB/s | 31 kB 00:00 2024-09-17T23:27:14.217 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T23:27:14.244 INFO:teuthology.orchestra.run.smithi037.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2024-09-17T23:27:14.269 INFO:teuthology.orchestra.run.smithi037.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2024-09-17T23:27:14.273 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:14.273 INFO:teuthology.orchestra.run.smithi037.stdout:Total 10 MB/s | 8.7 MB 00:00 2024-09-17T23:27:14.323 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:14.341 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:14.341 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:14.384 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:14.568 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:14.568 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:14.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T23:27:14.803 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:14.935 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T23:27:14.937 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout:Install 12 Packages 2024-09-17T23:27:14.938 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:14.939 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 8.7 M 2024-09-17T23:27:14.939 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 104 M 2024-09-17T23:27:14.939 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:15.134 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:15.213 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T23:27:15.213 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:15.247 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T23:27:15.295 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T23:27:15.324 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T23:27:15.351 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:15.352 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:15.353 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:15.353 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:27:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:15.353 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 145 k 2024-09-17T23:27:15.353 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 435 k 2024-09-17T23:27:15.353 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:15.437 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T23:27:15.447 INFO:teuthology.orchestra.run.smithi139.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 185 kB/s | 32 kB 00:00 2024-09-17T23:27:15.473 INFO:teuthology.orchestra.run.smithi139.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 83 kB/s | 16 kB 00:00 2024-09-17T23:27:15.555 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 721 kB/s | 145 kB 00:00 2024-09-17T23:27:15.555 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:15.556 INFO:teuthology.orchestra.run.smithi031.stdout:Total 716 kB/s | 145 kB 00:00 2024-09-17T23:27:15.556 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:15.564 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:15.564 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:15.573 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T23:27:15.615 INFO:teuthology.orchestra.run.smithi139.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-17T23:27:15.624 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:15.625 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:15.664 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T23:27:15.665 INFO:teuthology.orchestra.run.smithi139.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-09-17T23:27:15.713 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T23:27:15.732 INFO:teuthology.orchestra.run.smithi139.stdout:(5/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 7.8 MB/s | 3.6 MB 00:00 2024-09-17T23:27:15.740 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T23:27:15.772 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T23:27:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:15.791 INFO:teuthology.orchestra.run.smithi139.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 343 kB/s | 19 kB 00:00 2024-09-17T23:27:15.816 INFO:teuthology.orchestra.run.smithi139.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.6 MB/s | 548 kB 00:00 2024-09-17T23:27:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:15.858 INFO:teuthology.orchestra.run.smithi139.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-09-17T23:27:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:15.874 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:15.874 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T23:27:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:15.883 INFO:teuthology.orchestra.run.smithi139.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-17T23:27:15.925 INFO:teuthology.orchestra.run.smithi139.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-17T23:27:16.084 INFO:teuthology.orchestra.run.smithi139.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.0 MB/s | 2.0 MB 00:00 2024-09-17T23:27:16.642 INFO:teuthology.orchestra.run.smithi139.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 841 kB/s | 863 kB 00:01 2024-09-17T23:27:16.647 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:16.647 INFO:teuthology.orchestra.run.smithi139.stdout:Total 5.1 MB/s | 8.7 MB 00:01 2024-09-17T23:27:16.700 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:16.719 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:16.719 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:16.815 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T23:27:16.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:16.815 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:16.815 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:16.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:16.815 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:16.891 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T23:27:16.922 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:16.922 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T23:27:16.923 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:17.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-09-17T23:27:17.452 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:17.542 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T23:27:17.586 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T23:27:17.632 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T23:27:17.663 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T23:27:17.687 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:17.765 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T23:27:17.779 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:17.824 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:17.825 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:27:17.825 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:17.898 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T23:27:17.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T23:27:17.983 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T23:27:18.026 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T23:27:18.063 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T23:27:18.096 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T23:27:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T23:27:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T23:27:18.508 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:18.578 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:18.579 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:18.645 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:27:18.646 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-17T23:27:18.647 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:18.648 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 8.7 M 2024-09-17T23:27:18.648 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 104 M 2024-09-17T23:27:18.648 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:18.798 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T23:27:19.161 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T23:27:19.175 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 216 kB/s | 32 kB 00:00 2024-09-17T23:27:19.192 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 100 kB/s | 16 kB 00:00 2024-09-17T23:27:19.195 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T23:27:19.309 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-17T23:27:19.375 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-17T23:27:19.409 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:08 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:19.434 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 8.8 MB/s | 3.6 MB 00:00 2024-09-17T23:27:19.484 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 463 kB/s | 19 kB 00:00 2024-09-17T23:27:19.526 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.6 MB/s | 548 kB 00:00 2024-09-17T23:27:19.551 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:19.552 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.5 MB/s | 863 kB 00:00 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout:====================================================================================== 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout:====================================================================================== 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T23:27:19.553 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout:====================================================================================== 2024-09-17T23:27:19.554 INFO:teuthology.orchestra.run.smithi037.stdout:Install 12 Packages 2024-09-17T23:27:19.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:19.555 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 38 M 2024-09-17T23:27:19.555 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 212 M 2024-09-17T23:27:19.555 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:19.577 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-17T23:27:19.619 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 460 kB/s | 31 kB 00:00 2024-09-17T23:27:19.644 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-09-17T23:27:19.861 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.4 MB/s | 2.0 MB 00:00 2024-09-17T23:27:19.865 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:19.866 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.1 MB/s | 8.7 MB 00:01 2024-09-17T23:27:19.920 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:19.940 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:19.940 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:20.026 INFO:teuthology.orchestra.run.smithi037.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-17T23:27:20.060 INFO:teuthology.orchestra.run.smithi037.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-17T23:27:20.168 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:20.168 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:20.193 INFO:teuthology.orchestra.run.smithi037.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-09-17T23:27:20.294 INFO:teuthology.orchestra.run.smithi037.stdout:(4/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-17T23:27:20.361 INFO:teuthology.orchestra.run.smithi037.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 109 kB/s | 18 kB 00:00 2024-09-17T23:27:20.427 INFO:teuthology.orchestra.run.smithi037.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 289 kB/s | 38 kB 00:00 2024-09-17T23:27:20.511 INFO:teuthology.orchestra.run.smithi037.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.6 MB/s | 3.0 MB 00:00 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T23:27:20.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T23:27:20.542 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T23:27:20.542 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T23:27:20.670 INFO:teuthology.orchestra.run.smithi037.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-17T23:27:20.695 INFO:teuthology.orchestra.run.smithi037.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 916 kB/s | 245 kB 00:00 2024-09-17T23:27:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:20.728 INFO:teuthology.orchestra.run.smithi037.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.2 MB/s | 250 kB 00:00 2024-09-17T23:27:20.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T23:27:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T23:27:20.867 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T23:27:20.868 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T23:27:20.869 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:20.869 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T23:27:20.904 INFO:teuthology.orchestra.run.smithi037.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.1 MB/s | 4.9 MB 00:00 2024-09-17T23:27:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T23:27:21.034 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T23:27:21.095 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T23:27:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T23:27:21.221 INFO:teuthology.orchestra.run.smithi037.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-17T23:27:21.222 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:21.222 INFO:teuthology.orchestra.run.smithi037.stdout:Total 23 MB/s | 38 MB 00:01 2024-09-17T23:27:21.263 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T23:27:21.321 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T23:27:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T23:27:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T23:27:21.589 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:21.617 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:21.617 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:21.670 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:21.801 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T23:27:21.803 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-17T23:27:21.804 INFO:teuthology.orchestra.run.smithi139.stdout:Install 12 Packages 2024-09-17T23:27:21.805 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:21.805 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 38 M 2024-09-17T23:27:21.805 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 212 M 2024-09-17T23:27:21.805 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:22.074 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:22.075 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:22.374 INFO:teuthology.orchestra.run.smithi139.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-17T23:27:22.475 INFO:teuthology.orchestra.run.smithi139.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-09-17T23:27:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T23:27:22.500 INFO:teuthology.orchestra.run.smithi139.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-17T23:27:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T23:27:22.583 INFO:teuthology.orchestra.run.smithi139.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-09-17T23:27:22.667 INFO:teuthology.orchestra.run.smithi139.stdout:(5/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-17T23:27:22.692 INFO:teuthology.orchestra.run.smithi139.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 355 kB/s | 38 kB 00:00 2024-09-17T23:27:22.801 INFO:teuthology.orchestra.run.smithi139.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-17T23:27:22.869 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:22.901 INFO:teuthology.orchestra.run.smithi139.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-09-17T23:27:22.937 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T23:27:22.986 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T23:27:22.993 INFO:teuthology.orchestra.run.smithi139.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2024-09-17T23:27:23.018 INFO:teuthology.orchestra.run.smithi139.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-17T23:27:23.091 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T23:27:23.102 INFO:teuthology.orchestra.run.smithi139.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-17T23:27:23.129 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T23:27:23.321 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T23:27:23.352 INFO:teuthology.orchestra.run.smithi139.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-09-17T23:27:23.353 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T23:27:23.353 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:23.354 INFO:teuthology.orchestra.run.smithi139.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-17T23:27:23.423 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T23:27:23.709 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:23.736 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:23.736 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T23:27:23.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T23:27:23.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T23:27:24.118 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T23:27:24.168 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:24.168 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:24.237 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:24.250 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:24.251 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:24.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T23:27:24.965 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:25.064 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T23:27:25.083 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:25.108 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T23:27:25.207 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T23:27:25.222 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:25.223 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-17T23:27:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:25.223 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-17T23:27:25.223 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T23:27:25.223 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:25.224 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:25.225 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-17T23:27:25.225 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-17T23:27:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:25.225 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-09-17T23:27:25.225 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 212 M 2024-09-17T23:27:25.225 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:25.243 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T23:27:25.426 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T23:27:25.467 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T23:27:25.535 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-17T23:27:25.543 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T23:27:25.576 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T23:27:25.585 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-17T23:27:25.669 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-09-17T23:27:25.719 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-17T23:27:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-17T23:27:25.894 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-33 15 MB/s | 7.4 MB 00:00 2024-09-17T23:27:26.003 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2024-09-17T23:27:26.162 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-17T23:27:26.179 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T23:27:26.187 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 838 kB/s | 245 kB 00:00 2024-09-17T23:27:26.220 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.2 MB/s | 250 kB 00:00 2024-09-17T23:27:26.287 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.5 MB/s | 4.9 MB 00:00 2024-09-17T23:27:26.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T23:27:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T23:27:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T23:27:27.514 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T23:27:27.737 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-17T23:27:27.738 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:27.738 INFO:teuthology.orchestra.run.smithi031.stdout:Total 15 MB/s | 38 MB 00:02 2024-09-17T23:27:28.100 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:28.128 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:28.128 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:28.530 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T23:27:28.557 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T23:27:28.590 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:28.591 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T23:27:29.281 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T23:27:29.282 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T23:27:29.282 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T23:27:29.380 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:29.445 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T23:27:29.486 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T23:27:29.589 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T23:27:29.637 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T23:27:29.751 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:29.752 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:29.839 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T23:27:29.869 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T23:27:29.965 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T23:27:30.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-rook 2024-09-17T23:27:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T23:27:30.720 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:30.794 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T23:27:30.859 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:30.860 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T23:27:30.861 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout:Install 13 Packages 2024-09-17T23:27:30.862 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:30.863 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 2.2 M 2024-09-17T23:27:30.863 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 26 M 2024-09-17T23:27:30.863 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T23:27:31.115 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T23:27:31.116 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T23:27:31.116 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T23:27:31.116 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T23:27:31.368 INFO:teuthology.orchestra.run.smithi037.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 220 kB/s | 26 kB 00:00 2024-09-17T23:27:31.394 INFO:teuthology.orchestra.run.smithi037.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 136 kB/s | 19 kB 00:00 2024-09-17T23:27:31.419 INFO:teuthology.orchestra.run.smithi037.stdout:(3/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 296 kB/s | 49 kB 00:00 2024-09-17T23:27:31.511 INFO:teuthology.orchestra.run.smithi037.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-17T23:27:31.536 INFO:teuthology.orchestra.run.smithi037.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-17T23:27:31.561 INFO:teuthology.orchestra.run.smithi037.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T23:27:31.572 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T23:27:31.572 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:31.572 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:31.572 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:31.572 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:31.573 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:31.586 INFO:teuthology.orchestra.run.smithi037.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 641 kB/s | 32 kB 00:00 2024-09-17T23:27:31.603 INFO:teuthology.orchestra.run.smithi037.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 353 kB/s | 14 kB 00:00 2024-09-17T23:27:31.629 INFO:teuthology.orchestra.run.smithi037.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 5.4 MB/s | 220 kB 00:00 2024-09-17T23:27:31.654 INFO:teuthology.orchestra.run.smithi037.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-17T23:27:31.679 INFO:teuthology.orchestra.run.smithi037.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-17T23:27:31.738 INFO:teuthology.orchestra.run.smithi037.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2024-09-17T23:27:31.796 INFO:teuthology.orchestra.run.smithi037.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-09-17T23:27:31.799 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:31.799 INFO:teuthology.orchestra.run.smithi037.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-17T23:27:31.868 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:31.884 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:31.884 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:31.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-rook 2024-09-17T23:27:31.982 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:31.982 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T23:27:32.514 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:32.515 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:16 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:32.648 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:27:32.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T23:27:32.650 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:32.651 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:32.651 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:32.651 INFO:teuthology.orchestra.run.smithi139.stdout:Install 13 Packages 2024-09-17T23:27:32.651 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:32.651 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 2.2 M 2024-09-17T23:27:32.651 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 26 M 2024-09-17T23:27:32.652 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:32.688 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T23:27:32.742 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T23:27:32.796 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T23:27:32.832 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T23:27:32.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T23:27:32.981 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T23:27:33.091 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T23:27:33.143 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T23:27:33.219 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T23:27:33.243 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T23:27:33.441 INFO:teuthology.orchestra.run.smithi139.stdout:(1/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 308 kB/s | 49 kB 00:00 2024-09-17T23:27:33.783 INFO:teuthology.orchestra.run.smithi139.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 38 kB/s | 19 kB 00:00 2024-09-17T23:27:33.883 INFO:teuthology.orchestra.run.smithi139.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 43 kB/s | 26 kB 00:00 2024-09-17T23:27:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T23:27:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T23:27:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T23:27:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T23:27:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T23:27:34.208 INFO:teuthology.orchestra.run.smithi139.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 290 kB/s | 222 kB 00:00 2024-09-17T23:27:34.667 INFO:teuthology.orchestra.run.smithi139.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 180 kB/s | 159 kB 00:00 2024-09-17T23:27:34.708 INFO:teuthology.orchestra.run.smithi139.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 338 kB/s | 279 kB 00:00 2024-09-17T23:27:34.742 INFO:teuthology.orchestra.run.smithi139.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 431 kB/s | 32 kB 00:00 2024-09-17T23:27:34.776 INFO:teuthology.orchestra.run.smithi139.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 218 kB/s | 14 kB 00:00 2024-09-17T23:27:34.826 INFO:teuthology.orchestra.run.smithi139.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.6 MB/s | 220 kB 00:00 2024-09-17T23:27:34.851 INFO:teuthology.orchestra.run.smithi139.stdout:(10/13): python3-requests-oauthlib-1.3.0-12.el9 84 kB/s | 54 kB 00:00 2024-09-17T23:27:34.876 INFO:teuthology.orchestra.run.smithi139.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-17T23:27:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T23:27:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T23:27:34.943 INFO:teuthology.orchestra.run.smithi139.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 978 kB/s | 90 kB 00:00 2024-09-17T23:27:35.044 INFO:teuthology.orchestra.run.smithi139.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2024-09-17T23:27:35.048 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:35.048 INFO:teuthology.orchestra.run.smithi139.stdout:Total 946 kB/s | 2.2 MB 00:02 2024-09-17T23:27:35.118 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:35.133 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:35.133 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:35.224 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:35.225 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T23:27:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T23:27:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:35.242 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:35.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T23:27:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T23:27:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T23:27:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T23:27:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T23:27:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T23:27:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T23:27:35.926 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T23:27:35.987 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T23:27:36.042 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T23:27:36.078 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T23:27:36.102 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:25 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:36.119 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T23:27:36.207 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T23:27:36.208 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T23:27:36.209 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T23:27:36.209 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T23:27:36.209 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:36.209 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:36.239 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T23:27:36.243 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:36.243 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:36.243 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T23:27:36.244 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T23:27:36.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:36.245 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:36.245 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:36.245 INFO:teuthology.orchestra.run.smithi037.stdout:Install 6 Packages 2024-09-17T23:27:36.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:36.245 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 7.0 M 2024-09-17T23:27:36.246 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 33 M 2024-09-17T23:27:36.246 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:36.379 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T23:27:36.439 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T23:27:36.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-09-17T23:27:36.973 INFO:teuthology.orchestra.run.smithi037.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-17T23:27:36.998 INFO:teuthology.orchestra.run.smithi037.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 627 kB/s | 146 kB 00:00 2024-09-17T23:27:37.065 INFO:teuthology.orchestra.run.smithi037.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 867 kB/s | 58 kB 00:00 2024-09-17T23:27:37.148 INFO:teuthology.orchestra.run.smithi037.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.1 MB/s | 548 kB 00:00 2024-09-17T23:27:37.167 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T23:27:37.193 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:21 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:37.227 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T23:27:37.261 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T23:27:37.318 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T23:27:37.331 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:37.332 INFO:teuthology.orchestra.run.smithi037.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 279 kB/s | 74 kB 00:00 2024-09-17T23:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:37.332 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T23:27:37.333 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:37.334 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-09-17T23:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-09-17T23:27:37.335 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:37.347 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T23:27:37.440 INFO:teuthology.orchestra.run.smithi037.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 8.8 MB/s | 6.0 MB 00:00 2024-09-17T23:27:37.442 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:37.443 INFO:teuthology.orchestra.run.smithi037.stdout:Total 5.9 MB/s | 7.0 MB 00:01 2024-09-17T23:27:37.518 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:37.531 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:37.531 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:37.602 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 212 kB/s | 19 kB 00:00 2024-09-17T23:27:37.626 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 221 kB/s | 26 kB 00:00 2024-09-17T23:27:37.637 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:37.639 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:37.685 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 283 kB/s | 49 kB 00:00 2024-09-17T23:27:37.727 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-17T23:27:37.753 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-17T23:27:37.777 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T23:27:37.828 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 429 kB/s | 32 kB 00:00 2024-09-17T23:27:37.853 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 187 kB/s | 14 kB 00:00 2024-09-17T23:27:37.878 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-09-17T23:27:37.928 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-17T23:27:37.954 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-09-17T23:27:37.959 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:37.979 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T23:27:38.063 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.9 MB/s | 1.0 MB 00:00 2024-09-17T23:27:38.066 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:38.066 INFO:teuthology.orchestra.run.smithi031.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T23:27:38.091 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T23:27:38.092 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T23:27:38.092 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T23:27:38.092 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T23:27:38.092 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T23:27:38.124 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T23:27:38.133 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:38.149 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:38.149 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:38.177 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T23:27:38.249 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:38.249 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T23:27:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:38.431 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:38.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T23:27:38.753 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T23:27:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:38.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T23:27:38.912 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T23:27:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T23:27:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T23:27:39.025 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T23:27:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T23:27:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T23:27:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T23:27:39.256 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:23 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T23:27:39.388 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:39.388 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:39.388 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:39.388 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout:Install 6 Packages 2024-09-17T23:27:39.389 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:39.390 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 7.0 M 2024-09-17T23:27:39.390 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 33 M 2024-09-17T23:27:39.390 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:39.399 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T23:27:39.459 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T23:27:39.695 INFO:teuthology.orchestra.run.smithi139.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-17T23:27:39.759 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T23:27:39.759 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T23:27:39.759 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T23:27:39.759 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T23:27:39.759 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T23:27:39.759 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T23:27:39.762 INFO:teuthology.orchestra.run.smithi139.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 711 kB/s | 146 kB 00:00 2024-09-17T23:27:39.820 INFO:teuthology.orchestra.run.smithi139.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 994 kB/s | 58 kB 00:00 2024-09-17T23:27:39.912 INFO:teuthology.orchestra.run.smithi139.stdout:(4/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-17T23:27:39.937 INFO:teuthology.orchestra.run.smithi139.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.2 MB/s | 548 kB 00:00 2024-09-17T23:27:39.963 INFO:teuthology.orchestra.run.smithi139.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 524 kB/s | 74 kB 00:00 2024-09-17T23:27:39.966 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:39.966 INFO:teuthology.orchestra.run.smithi139.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-17T23:27:40.038 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T23:27:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:40.048 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T23:27:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T23:27:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T23:27:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T23:27:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T23:27:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:40.049 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:40.050 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:40.050 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:40.151 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:40.151 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:40.218 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T23:27:40.273 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T23:27:40.307 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-fuse 2024-09-17T23:27:40.307 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T23:27:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T23:27:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T23:27:40.458 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:40.635 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T23:27:40.698 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T23:27:40.907 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:29 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:41.045 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-17T23:27:41.046 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout:Install 2 Packages 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 977 k 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 2.9 M 2024-09-17T23:27:41.047 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T23:27:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T23:27:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T23:27:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T23:27:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T23:27:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T23:27:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T23:27:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T23:27:41.216 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T23:27:41.301 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T23:27:41.342 INFO:teuthology.orchestra.run.smithi037.stdout:(1/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.5 MB/s | 897 kB 00:00 2024-09-17T23:27:41.372 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T23:27:41.410 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T23:27:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T23:27:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:41.505 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T23:27:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:41.507 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:41.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T23:27:41.767 INFO:teuthology.orchestra.run.smithi037.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 118 kB/s | 80 kB 00:00 2024-09-17T23:27:41.768 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:41.769 INFO:teuthology.orchestra.run.smithi037.stdout:Total 1.3 MB/s | 977 kB 00:00 2024-09-17T23:27:41.781 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:41.805 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:41.806 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:41.893 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:41.894 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:42.089 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:42.185 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T23:27:42.185 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T23:27:42.185 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T23:27:42.185 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T23:27:42.185 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T23:27:42.185 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T23:27:42.238 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T23:27:42.312 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T23:27:42.372 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:26 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:42.446 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T23:27:42.446 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:42.447 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:42.509 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T23:27:42.510 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-09-17T23:27:42.511 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:42.512 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-09-17T23:27:42.512 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-09-17T23:27:42.512 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:42.687 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-fuse 2024-09-17T23:27:43.015 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T23:27:43.015 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T23:27:43.039 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 668 kB/s | 146 kB 00:00 2024-09-17T23:27:43.148 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.0 MB/s | 548 kB 00:00 2024-09-17T23:27:43.182 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-09-17T23:27:43.215 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-17T23:27:43.265 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:27 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T23:27:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:43.319 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T23:27:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:43.319 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:43.324 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 493 kB/s | 248 kB 00:00 2024-09-17T23:27:43.398 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:43.399 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:43.400 INFO:teuthology.orchestra.run.smithi139.stdout:Install 2 Packages 2024-09-17T23:27:43.400 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:43.400 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 977 k 2024-09-17T23:27:43.400 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 2.9 M 2024-09-17T23:27:43.400 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:43.573 INFO:teuthology.orchestra.run.smithi139.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 525 kB/s | 80 kB 00:00 2024-09-17T23:27:43.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-volume 2024-09-17T23:27:43.698 INFO:teuthology.orchestra.run.smithi139.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.2 MB/s | 897 kB 00:00 2024-09-17T23:27:43.699 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:43.699 INFO:teuthology.orchestra.run.smithi139.stdout:Total 3.2 MB/s | 977 kB 00:00 2024-09-17T23:27:43.711 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:43.732 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:43.732 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:43.815 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:43.815 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:44.054 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:44.199 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T23:27:44.217 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:44.272 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T23:27:44.356 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:44.357 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:44.357 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-17T23:27:44.357 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 285 k 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 1.3 M 2024-09-17T23:27:44.358 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:44.557 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-17T23:27:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:Total 3.4 MB/s | 7.0 MB 00:02 2024-09-17T23:27:44.588 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T23:27:44.588 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:44.589 INFO:teuthology.orchestra.run.smithi037.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T23:27:44.589 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:44.598 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:44.598 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:44.625 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:44.625 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:44.636 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:44.649 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:44.755 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:44.756 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:44.852 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:44.915 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:44.932 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:44.932 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:44.932 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T23:27:44.932 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:44.991 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T23:27:44.991 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T23:27:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T23:27:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T23:27:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:45.259 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T23:27:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:45.259 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:45.310 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T23:27:45.558 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-volume 2024-09-17T23:27:45.878 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T23:27:45.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T23:27:46.000 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:46.001 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:46.001 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:46.001 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:46.001 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:46.001 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:46.045 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T23:27:46.071 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T23:27:46.162 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:46.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install librados-devel 2024-09-17T23:27:46.300 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:46.300 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:46.300 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-17T23:27:46.300 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:46.301 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 285 k 2024-09-17T23:27:46.302 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 1.3 M 2024-09-17T23:27:46.302 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:46.536 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T23:27:46.536 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:46.537 INFO:teuthology.orchestra.run.smithi139.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T23:27:46.537 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:46.545 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:46.546 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:46.569 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:46.570 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:46.793 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:46.798 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T23:27:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T23:27:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T23:27:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T23:27:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T23:27:46.799 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T23:27:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:46.879 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:46.879 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:46.879 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T23:27:46.879 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:46.904 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:47.041 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:47.042 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-09-17T23:27:47.042 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:47.042 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T23:27:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 127 k 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 456 k 2024-09-17T23:27:47.043 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T23:27:47.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:47.087 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:47.088 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:47.226 INFO:teuthology.orchestra.run.smithi037.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 694 kB/s | 127 kB 00:00 2024-09-17T23:27:47.226 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:47.227 INFO:teuthology.orchestra.run.smithi037.stdout:Total 689 kB/s | 127 kB 00:00 2024-09-17T23:27:47.227 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:47.232 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:47.233 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:47.301 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:47.301 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:47.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-09-17T23:27:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:47.920 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:47.920 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:47.920 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:47.920 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:47.921 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:47.921 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:47.939 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:31 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:48.005 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:48.077 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:48.077 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-09-17T23:27:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:48.079 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 977 k 2024-09-17T23:27:48.079 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.9 M 2024-09-17T23:27:48.079 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:48.190 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install librados-devel 2024-09-17T23:27:48.259 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:48.259 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:48.259 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:48.259 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:48.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:48.260 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:48.392 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 549 kB/s | 80 kB 00:00 2024-09-17T23:27:48.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install libcephfs2 2024-09-17T23:27:48.551 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 2.9 MB/s | 897 kB 00:00 2024-09-17T23:27:48.551 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:48.552 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.0 MB/s | 977 kB 00:00 2024-09-17T23:27:48.564 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:48.587 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:48.588 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:48.675 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:48.675 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:48.791 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:48.926 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:48.926 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-17T23:27:48.926 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 127 k 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 456 k 2024-09-17T23:27:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T23:27:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T23:27:49.107 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:49.132 INFO:teuthology.orchestra.run.smithi139.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 618 kB/s | 127 kB 00:00 2024-09-17T23:27:49.133 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:49.133 INFO:teuthology.orchestra.run.smithi139.stdout:Total 613 kB/s | 127 kB 00:00 2024-09-17T23:27:49.134 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:49.139 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:49.139 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:49.200 INFO:teuthology.orchestra.run.smithi037.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:49.202 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:49.202 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:49.245 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:49.246 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:49.246 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:49.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install libcephfs-devel 2024-09-17T23:27:49.349 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:49.463 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T23:27:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T23:27:49.913 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:49.937 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:50.074 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:50.074 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:50.074 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 32 k 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 142 k 2024-09-17T23:27:50.075 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:50.096 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T23:27:50.096 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:50.096 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:50.097 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T23:27:50.097 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:50.097 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:50.182 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:50.182 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:50.237 INFO:teuthology.orchestra.run.smithi037.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 196 kB/s | 32 kB 00:00 2024-09-17T23:27:50.237 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:Total 195 kB/s | 32 kB 00:00 2024-09-17T23:27:50.238 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:50.241 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:50.261 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:50.261 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:50.355 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:50.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-volume 2024-09-17T23:27:50.426 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install libcephfs2 2024-09-17T23:27:50.470 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:50.870 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:50.989 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:51.026 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:51.103 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:51.103 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:51.103 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:51.103 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:51.104 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:51.104 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:51.119 INFO:teuthology.orchestra.run.smithi139.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:51.128 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:51.128 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 285 k 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.3 M 2024-09-17T23:27:51.129 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:51.164 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:51.166 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:51.166 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:51.240 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install libcephfs-devel 2024-09-17T23:27:51.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install librados2 2024-09-17T23:27:51.363 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T23:27:51.363 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:51.364 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T23:27:51.364 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:51.373 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:51.373 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:51.399 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:51.400 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:51.629 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:51.699 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:51.716 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:27:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T23:27:51.717 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:51.826 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:51.893 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:51.960 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:51.961 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:51.962 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:51.962 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:27:51.962 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:51.962 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 32 k 2024-09-17T23:27:51.962 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 142 k 2024-09-17T23:27:51.962 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:51.985 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:52.031 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:52.032 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:52.033 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:52.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install librbd1 2024-09-17T23:27:52.119 INFO:teuthology.orchestra.run.smithi139.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 203 kB/s | 32 kB 00:00 2024-09-17T23:27:52.119 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:52.120 INFO:teuthology.orchestra.run.smithi139.stdout:Total 201 kB/s | 32 kB 00:00 2024-09-17T23:27:52.120 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:52.123 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:52.123 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:52.137 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:52.137 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:52.224 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:52.358 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:52.700 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:52.728 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T23:27:52.728 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:52.728 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:52.728 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T23:27:52.729 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:52.729 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:52.791 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:52.837 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:52.838 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:52.838 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:52.906 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-rados 2024-09-17T23:27:52.959 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:52.960 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:52.960 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:52.960 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:52.960 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:52.960 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:53.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-09-17T23:27:53.148 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install librados2 2024-09-17T23:27:53.517 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:53.610 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:53.654 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:53.655 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:53.656 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:53.657 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:53.723 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-rgw 2024-09-17T23:27:53.729 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:53.791 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:53.792 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:53.792 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:53.793 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:27:53.794 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:53.794 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-09-17T23:27:53.794 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-09-17T23:27:53.794 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:53.819 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:53.863 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:53.864 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:53.864 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:53.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install librbd1 2024-09-17T23:27:54.001 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 609 kB/s | 127 kB 00:00 2024-09-17T23:27:54.001 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:54.002 INFO:teuthology.orchestra.run.smithi031.stdout:Total 605 kB/s | 127 kB 00:00 2024-09-17T23:27:54.002 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:54.008 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:54.008 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:54.075 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:54.075 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:54.229 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:54.327 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:54.336 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:54.429 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:54.475 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:54.476 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:54.477 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:54.509 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:54.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-cephfs 2024-09-17T23:27:54.599 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:54.642 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:54.643 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:54.643 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:54.713 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-rados 2024-09-17T23:27:54.752 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:55.017 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:55.018 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:55.018 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:55.018 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:55.147 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:55.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-09-17T23:27:55.240 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:55.285 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:55.286 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:55.286 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:55.299 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:55.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install python3-rbd 2024-09-17T23:27:55.387 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:55.431 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:55.431 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:55.432 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:55.495 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-rgw 2024-09-17T23:27:55.856 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:55.949 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:55.964 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:55.995 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:55.997 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:27:55.997 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:56.057 INFO:teuthology.orchestra.run.smithi037.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:56.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-09-17T23:27:56.083 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:56.102 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:56.104 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:27:56.104 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:56.173 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:56.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-fuse 2024-09-17T23:27:56.216 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:56.217 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:56.217 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:56.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-cephfs 2024-09-17T23:27:56.670 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:56.775 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:56.808 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:27:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:56.810 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 32 k 2024-09-17T23:27:56.810 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 142 k 2024-09-17T23:27:56.810 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:27:56.869 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:56.914 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:56.915 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 85 k 2024-09-17T23:27:56.916 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 231 k 2024-09-17T23:27:56.916 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:56.958 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:56.970 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 197 kB/s | 32 kB 00:00 2024-09-17T23:27:56.971 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:56.971 INFO:teuthology.orchestra.run.smithi031.stdout:Total 196 kB/s | 32 kB 00:00 2024-09-17T23:27:56.971 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:27:56.975 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:27:56.975 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:27:56.988 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:27:56.989 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:27:57.001 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:57.002 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:57.002 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:57.074 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:27:57.083 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install python3-rbd 2024-09-17T23:27:57.098 INFO:teuthology.orchestra.run.smithi037.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 469 kB/s | 85 kB 00:00 2024-09-17T23:27:57.098 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:57.099 INFO:teuthology.orchestra.run.smithi037.stdout:Total 465 kB/s | 85 kB 00:00 2024-09-17T23:27:57.099 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:57.106 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:57.107 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:57.173 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:57.174 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:57.188 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:57.335 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:27:57.424 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:57.584 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:57.664 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:57.754 INFO:teuthology.orchestra.run.smithi139.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:57.797 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:57.798 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:27:57.798 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:57.825 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:57.827 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:57.827 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:27:57.828 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:57.828 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:27:57.828 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:57.866 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-fuse 2024-09-17T23:27:58.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-09-17T23:27:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:58.096 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:27:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:58.096 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:27:58.367 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-mirror 2024-09-17T23:27:58.454 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:58.594 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:58.595 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:27:58.596 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:27:58.596 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:27:58.596 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:58.596 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 85 k 2024-09-17T23:27:58.596 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 231 k 2024-09-17T23:27:58.596 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:27:58.611 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:58.705 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:58.751 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:27:58.752 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:58.789 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 444 kB/s | 85 kB 00:00 2024-09-17T23:27:58.790 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:58.790 INFO:teuthology.orchestra.run.smithi139.stdout:Total 439 kB/s | 85 kB 00:00 2024-09-17T23:27:58.790 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:27:58.798 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:27:58.798 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:27:58.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-09-17T23:27:58.864 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:27:58.864 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:27:58.986 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:27:59.029 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:27:59.127 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:27:59.128 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:59.128 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repo Size 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 3.1 M 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 13 M 2024-09-17T23:27:59.129 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:27:59.130 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:59.430 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:27:59.451 INFO:teuthology.orchestra.run.smithi037.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.6 MB/s | 3.1 MB 00:00 2024-09-17T23:27:59.451 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:27:59.452 INFO:teuthology.orchestra.run.smithi037.stdout:Total 9.6 MB/s | 3.1 MB 00:00 2024-09-17T23:27:59.452 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:27:59.462 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:27:59.463 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:27:59.513 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:59.523 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:27:59.549 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:27:59.549 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:27:59.569 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:27:59.571 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:27:59.571 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:27:59.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-09-17T23:27:59.777 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:27:59.778 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:59.778 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:27:59.778 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:27:59.778 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:27:59.778 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:27:59.997 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:28:00.052 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:00.074 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-mirror 2024-09-17T23:28:00.079 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:00.080 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:28:00.080 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T23:28:00.080 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T23:28:00.080 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T23:28:00.080 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:00.253 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:00.346 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:00.392 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:00.393 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:28:00.393 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:00.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-09-17T23:28:00.673 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:00.810 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:28:00.811 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:28:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repo Size 2024-09-17T23:28:00.811 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:28:00.811 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:28:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 3.1 M 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 13 M 2024-09-17T23:28:00.812 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:28:01.072 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:01.164 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:01.175 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 8.5 MB/s | 3.1 MB 00:00 2024-09-17T23:28:01.176 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:28:01.176 INFO:teuthology.orchestra.run.smithi139.stdout:Total 8.5 MB/s | 3.1 MB 00:00 2024-09-17T23:28:01.176 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:28:01.185 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:28:01.185 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:28:01.209 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:01.210 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:28:01.211 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:01.213 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:01.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:01.213 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:28:01.213 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:01.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:01.213 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:28:01.266 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:28:01.266 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:28:01.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-09-17T23:28:01.430 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-nbd 2024-09-17T23:28:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:28:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:01.793 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:28:01.794 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T23:28:01.794 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T23:28:01.794 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T23:28:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:01.887 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:01.980 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:02.027 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:02.028 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:28:02.028 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:02.042 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:28:02.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-09-17T23:28:02.182 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:28:02.183 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:28:02.183 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-09-17T23:28:02.183 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:28:02.183 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout:Install 1 Package 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 170 k 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 494 k 2024-09-17T23:28:02.184 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-17T23:28:02.396 INFO:teuthology.orchestra.run.smithi037.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 805 kB/s | 170 kB 00:00 2024-09-17T23:28:02.396 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:28:02.397 INFO:teuthology.orchestra.run.smithi037.stdout:Total 799 kB/s | 170 kB 00:00 2024-09-17T23:28:02.397 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-17T23:28:02.405 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-17T23:28:02.405 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-17T23:28:02.473 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-17T23:28:02.474 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-17T23:28:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-17T23:28:02.707 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:02.801 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:02.847 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:02.849 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:28:02.849 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:02.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-09-17T23:28:03.002 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:03.002 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:03.003 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:28:03.003 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:03.003 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:03.003 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:28:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:03.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-nbd 2024-09-17T23:28:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:03.399 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-17T23:28:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:03.399 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:28:03.534 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:03.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install rbd-nbd 2024-09-17T23:28:03.675 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:28:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:03.677 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 85 k 2024-09-17T23:28:03.677 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 231 k 2024-09-17T23:28:03.677 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:28:03.826 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:03.865 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 454 kB/s | 85 kB 00:00 2024-09-17T23:28:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:28:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:Total 451 kB/s | 85 kB 00:00 2024-09-17T23:28:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:28:03.874 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:28:03.874 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:28:03.941 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:28:03.941 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:28:03.963 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-17T23:28:03.964 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-17T23:28:03.965 INFO:teuthology.orchestra.run.smithi139.stdout:Install 1 Package 2024-09-17T23:28:03.965 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:03.965 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 170 k 2024-09-17T23:28:03.965 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 494 k 2024-09-17T23:28:03.965 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-17T23:28:04.098 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:28:04.170 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 831 kB/s | 170 kB 00:00 2024-09-17T23:28:04.171 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:28:04.171 INFO:teuthology.orchestra.run.smithi139.stdout:Total 824 kB/s | 170 kB 00:00 2024-09-17T23:28:04.171 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-17T23:28:04.178 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-17T23:28:04.179 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-17T23:28:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:04.242 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-17T23:28:04.242 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-17T23:28:04.254 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 11:26:11 PM UTC. 2024-09-17T23:28:04.346 INFO:teuthology.orchestra.run.smithi037.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:04.394 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-17T23:28:04.395 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-09-17T23:28:04.395 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-17T23:28:04.403 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-17T23:28:04.471 DEBUG:teuthology.parallel:result is None 2024-09-17T23:28:04.517 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:04.608 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:04.865 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:28:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:04.866 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:04.933 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:05.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-09-17T23:28:05.209 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:05.209 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:05.210 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-17T23:28:05.210 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:05.210 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:05.210 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:28:05.465 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install rbd-nbd 2024-09-17T23:28:05.766 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:05.906 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:28:05.907 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:05.908 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.1 M 2024-09-17T23:28:05.908 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 13 M 2024-09-17T23:28:05.908 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:28:06.063 INFO:teuthology.orchestra.run.smithi139.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:06.158 INFO:teuthology.orchestra.run.smithi139.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:06.203 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-17T23:28:06.204 INFO:teuthology.orchestra.run.smithi139.stdout:Nothing to do. 2024-09-17T23:28:06.204 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-17T23:28:06.236 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.5 MB/s | 3.1 MB 00:00 2024-09-17T23:28:06.236 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:28:06.236 INFO:teuthology.orchestra.run.smithi031.stdout:Total 9.5 MB/s | 3.1 MB 00:00 2024-09-17T23:28:06.236 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:28:06.245 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:28:06.245 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:28:06.272 DEBUG:teuthology.parallel:result is None 2024-09-17T23:28:06.332 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:28:06.332 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:28:06.783 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:28:06.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:06.870 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T23:28:06.871 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T23:28:06.871 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T23:28:06.871 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T23:28:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:07.976 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:28:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:07.976 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:08.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-17T23:28:08.816 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:08.955 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 170 k 2024-09-17T23:28:08.957 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 494 k 2024-09-17T23:28:08.957 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-17T23:28:09.171 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 795 kB/s | 170 kB 00:00 2024-09-17T23:28:09.171 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-17T23:28:09.172 INFO:teuthology.orchestra.run.smithi031.stdout:Total 790 kB/s | 170 kB 00:00 2024-09-17T23:28:09.172 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-17T23:28:09.180 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-17T23:28:09.181 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-17T23:28:09.249 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-17T23:28:09.249 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-17T23:28:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-17T23:28:09.522 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:09.932 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T23:28:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-17T23:28:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T23:28:10.184 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:10.184 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:10.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-17T23:28:11.072 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:55 ago on Tue 17 Sep 2024 11:26:16 PM UTC. 2024-09-17T23:28:11.167 INFO:teuthology.orchestra.run.smithi031.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T23:28:11.214 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-17T23:28:11.215 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-17T23:28:11.215 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-17T23:28:11.289 DEBUG:teuthology.parallel:result is None 2024-09-17T23:28:11.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:28:11.435 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T23:28:11.457 INFO:teuthology.orchestra.run.smithi031.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T23:28:11.457 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T23:28:11.458 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T23:28:11.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:28:11.603 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T23:28:11.629 INFO:teuthology.orchestra.run.smithi037.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T23:28:11.629 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T23:28:11.630 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T23:28:11.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T23:28:11.775 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T23:28:11.800 INFO:teuthology.orchestra.run.smithi139.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T23:28:11.801 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T23:28:11.801 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T23:28:11.803 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T23:28:11.803 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:11.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T23:28:11.830 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:11.831 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T23:28:11.862 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:11.862 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T23:28:11.891 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T23:28:11.892 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:11.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T23:28:11.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T23:28:11.986 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:11.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T23:28:12.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T23:28:12.088 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:12.088 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T23:28:12.118 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T23:28:12.187 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T23:28:12.188 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:12.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T23:28:12.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T23:28:12.283 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:12.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T23:28:12.313 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T23:28:12.384 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:12.468 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T23:28:12.498 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T23:28:12.566 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T23:28:12.566 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:12.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T23:28:12.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T23:28:12.663 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:12.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T23:28:12.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T23:28:12.765 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:12.765 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T23:28:12.795 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T23:28:12.864 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T23:28:12.941 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T23:28:12.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-09-17T23:28:12.944 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2024-09-17T23:28:12.947 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2024-09-17T23:28:12.976 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T23:28:12.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T23:28:13.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T23:28:13.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T23:28:13.048 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T23:28:13.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T23:28:13.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T23:28:13.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T23:28:13.126 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T23:28:13.126 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T23:28:13.126 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909589', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLs8FF+xIy01YbyotcQ123T663AWzxzrKj5oV2BGsrHKszR17MqVQEBcYvHpx+oMEqirCAlL2z1AJXbhHXzIxqY=', 'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKs3Rm/2/C2sFChM6hV2e2V6ZuJcSxS7U0MysWAL+UuqNlrofYCPM9oCemhK8JQlG+0VN2uUps68eK4axEJBlXQ=', 'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgohmViWyCpuOwR5MJ8irE3Q36uxhMFz42p4fqWgsKZ5YZexH+y1QU53MPUGPG9xC1u1m2/W9Zr0Mgp41VmRM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T23:28:13.127 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T23:28:13.154 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T23:28:13.158 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T23:28:13.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T23:28:13.212 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T23:28:13.216 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T23:28:13.244 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:13.244 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T23:28:13.295 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T23:28:13.296 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-17 23:28:07.129427808 +0000 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-17 23:25:29.782047145 +0000 2024-09-17T23:28:13.353 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-17 23:25:29.782047145 +0000 2024-09-17T23:28:13.354 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-17 23:25:29.782047145 +0000 2024-09-17T23:28:13.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T23:28:13.418 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-17T23:28:13.418 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-17T23:28:13.419 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00021145 s, 2.4 MB/s 2024-09-17T23:28:13.420 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T23:28:13.477 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-17 23:28:07.129427808 +0000 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-17 23:25:30.126041708 +0000 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-17 23:25:30.126041708 +0000 2024-09-17T23:28:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-17 23:25:30.126041708 +0000 2024-09-17T23:28:13.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T23:28:13.600 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-17T23:28:13.601 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-17T23:28:13.601 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000198168 s, 2.6 MB/s 2024-09-17T23:28:13.602 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T23:28:13.659 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-17T23:28:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T23:28:13.716 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:13.716 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-17T23:28:13.717 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:13.717 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:13.717 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-17 23:28:07.130427792 +0000 2024-09-17T23:28:13.717 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-17 23:25:30.455036511 +0000 2024-09-17T23:28:13.717 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-17 23:25:30.455036511 +0000 2024-09-17T23:28:13.717 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-17 23:25:30.455036511 +0000 2024-09-17T23:28:13.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T23:28:13.785 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-17T23:28:13.785 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-17T23:28:13.785 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000207945 s, 2.5 MB/s 2024-09-17T23:28:13.785 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T23:28:13.841 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-17T23:28:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-17 23:28:07.130427792 +0000 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-17 23:25:30.777031425 +0000 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-17 23:25:30.777031425 +0000 2024-09-17T23:28:13.898 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-17 23:25:30.777031425 +0000 2024-09-17T23:28:13.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T23:28:13.961 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-17T23:28:13.962 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-17T23:28:13.962 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000233917 s, 2.2 MB/s 2024-09-17T23:28:13.963 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T23:28:14.020 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:14.020 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:14.020 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T23:28:14.040 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T23:28:14.040 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-09-17T23:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T23:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:14.099 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-17T23:28:14.100 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:14.100 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:14.100 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 23:28:00.342871335 +0000 2024-09-17T23:28:14.100 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 23:25:30.115554333 +0000 2024-09-17T23:28:14.100 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 23:25:30.115554333 +0000 2024-09-17T23:28:14.100 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 23:25:30.115554333 +0000 2024-09-17T23:28:14.100 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T23:28:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T23:28:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T23:28:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000280469 s, 1.8 MB/s 2024-09-17T23:28:14.170 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T23:28:14.230 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 23:28:00.342871335 +0000 2024-09-17T23:28:14.289 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 23:25:30.457550167 +0000 2024-09-17T23:28:14.290 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 23:25:30.457550167 +0000 2024-09-17T23:28:14.290 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 23:25:30.457550167 +0000 2024-09-17T23:28:14.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T23:28:14.358 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T23:28:14.358 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T23:28:14.358 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000251008 s, 2.0 MB/s 2024-09-17T23:28:14.360 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T23:28:14.419 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 23:28:00.342871335 +0000 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 23:25:30.789546124 +0000 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 23:25:30.789546124 +0000 2024-09-17T23:28:14.478 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 23:25:30.789546124 +0000 2024-09-17T23:28:14.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T23:28:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T23:28:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T23:28:14.545 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000189194 s, 2.7 MB/s 2024-09-17T23:28:14.546 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T23:28:14.606 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-09-17T23:28:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T23:28:14.663 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-17 23:28:00.343871323 +0000 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-17 23:25:31.119542105 +0000 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-17 23:25:31.119542105 +0000 2024-09-17T23:28:14.664 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-17 23:25:31.119542105 +0000 2024-09-17T23:28:14.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T23:28:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-17T23:28:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-17T23:28:14.731 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00023704 s, 2.2 MB/s 2024-09-17T23:28:14.733 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T23:28:14.792 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:14.792 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T23:28:14.792 INFO:tasks.ceph:Generating config... 2024-09-17T23:28:14.793 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T23:28:14.794 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T23:28:14.794 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T23:28:14.794 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T23:28:14.794 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T23:28:14.794 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T23:28:14.794 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T23:28:14.794 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T23:28:14.794 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T23:28:14.794 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-17T23:28:14.794 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-17T23:28:14.794 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T23:28:14.795 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T23:28:14.796 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T23:28:14.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T23:28:14.868 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T23:28:14.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T23:28:14.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T23:28:14.942 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'), ('mon.b', '172.21.15.37')] 2024-09-17T23:28:14.942 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.37', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T23:28:14.943 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:14.943 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T23:28:14.997 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --add b 172.21.15.37 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:15.084 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:15.084 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid b662004c-f841-43da-8884-84d1805b0cbb 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:fsid b662004c-f841-43da-8884-84d1805b0cbb 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-09-17T23:28:15.083481+0000 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-17T23:28:15.083481+0000 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 17 (quincy) 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.b 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-09-17T23:28:15.085 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T23:28:15.087 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T23:28:15.102 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b662004c-f841-43da-8884-84d1805b0cbb... 2024-09-17T23:28:15.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T23:28:15.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T23:28:15.148 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T23:28:15.187 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-17T23:28:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-09-17T23:28:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-09-17T23:28:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-09-17T23:28:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.37" 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: mon client directed command retry = 5 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: ms inject socket failures = 5000 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = b662004c-f841-43da-8884-84d1805b0cbb 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-09-17T23:28:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-09-17T23:28:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-09-17T23:28:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression algorithm = zstd 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression mode = aggressive 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-09-17T23:28:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-17T23:28:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T23:28:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T23:28:15.197 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.198 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T23:28:15.199 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.200 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.37" 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: mon client directed command retry = 5 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on pool no app = False 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: ms inject socket failures = 5000 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = b662004c-f841-43da-8884-84d1805b0cbb 2024-09-17T23:28:15.201 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-09-17T23:28:15.202 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.203 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore compression algorithm = zstd 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore compression mode = aggressive 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 1/20 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 1/20 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 4/10 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-09-17T23:28:15.204 INFO:teuthology.orchestra.run.smithi037.stdout: osd blocked scrub grace period = 3600 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: osd max markdown count = 1000 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-09-17T23:28:15.205 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T23:28:15.206 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-09-17T23:28:15.207 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-09-17T23:28:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T23:28:15.208 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T23:28:15.208 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-09-17T23:28:15.208 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-09-17T23:28:15.208 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-09-17T23:28:15.208 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.209 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-09-17T23:28:15.210 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.211 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.37" 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: mon client directed command retry = 5 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on pool no app = False 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: ms inject socket failures = 5000 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = b662004c-f841-43da-8884-84d1805b0cbb 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-09-17T23:28:15.212 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-09-17T23:28:15.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-09-17T23:28:15.216 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore compression algorithm = zstd 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore compression mode = aggressive 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 1/20 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 1/20 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 4/10 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: osd blocked scrub grace period = 3600 2024-09-17T23:28:15.217 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: osd max markdown count = 1000 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-09-17T23:28:15.218 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.219 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T23:28:15.220 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T23:28:15.221 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-09-17T23:28:15.221 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-09-17T23:28:15.221 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-09-17T23:28:15.222 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T23:28:15.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T23:28:15.318 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T23:28:15.318 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:15.318 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T23:28:15.334 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:15.334 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T23:28:15.390 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:28:15.390 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:15.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T23:28:15.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T23:28:15.466 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:15.466 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:15.521 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:28:15.522 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:15.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T23:28:15.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T23:28:15.565 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:15.565 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:15.623 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2024-09-17T23:28:15.623 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:15.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T23:28:15.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T23:28:15.665 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:15.665 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:15.722 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T23:28:15.722 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T23:28:15.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T23:28:15.803 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T23:28:15.806 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T23:28:15.807 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T23:28:15.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T23:28:15.863 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T23:28:15.877 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T23:28:15.877 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T23:28:15.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T23:28:15.903 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:15.903 INFO:tasks.ceph:role: osd.0 2024-09-17T23:28:15.903 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:28:15.904 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T23:28:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T23:28:15.977 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T23:28:15.977 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T23:28:15.977 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T23:28:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T23:28:15.978 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-17T23:28:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T23:28:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T23:28:15.978 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T23:28:15.978 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T23:28:16.484 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-17T23:28:16.486 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-17T23:28:16.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T23:28:16.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T23:28:16.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T23:28:16.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:16.742 INFO:tasks.ceph:role: osd.1 2024-09-17T23:28:16.742 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:28:16.742 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T23:28:16.817 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T23:28:17.260 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-17T23:28:17.262 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-17T23:28:17.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T23:28:17.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T23:28:17.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T23:28:17.449 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:17.449 INFO:tasks.ceph:role: osd.2 2024-09-17T23:28:17.449 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:28:17.449 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T23:28:17.521 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T23:28:17.522 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T23:28:18.013 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-17T23:28:18.015 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-17T23:28:18.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T23:28:18.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T23:28:18.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:18.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:18.294+0000 7f7ad2a20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T23:28:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:18.298+0000 7f7ad2a20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T23:28:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:18.298+0000 7f7ad2a20640 -1 bdev(0x561c01dd3800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T23:28:18.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:18.298+0000 7f7ad2a20640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T23:28:21.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T23:28:21.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:21.327+0000 7fec723d4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T23:28:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:21.328+0000 7fec723d4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T23:28:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:21.328+0000 7fec723d4640 -1 bdev(0x5603cc367800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T23:28:21.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:21.328+0000 7fec723d4640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T23:28:24.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T23:28:24.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:24.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:24.632+0000 7f9e9e814640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T23:28:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:24.632+0000 7f9e9e814640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T23:28:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:24.632+0000 7f9e9e814640 -1 bdev(0x55e6308eb800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T23:28:24.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:28:24.632+0000 7f9e9e814640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T23:28:27.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T23:28:27.586 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T23:28:27.618 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:27.618 INFO:tasks.ceph:role: osd.3 2024-09-17T23:28:27.618 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:28:27.618 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T23:28:27.696 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T23:28:27.696 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T23:28:27.696 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T23:28:27.697 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T23:28:28.201 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T23:28:28.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T23:28:28.204 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-17T23:28:28.349 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T23:28:28.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T23:28:28.448 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:28.448 INFO:tasks.ceph:role: osd.4 2024-09-17T23:28:28.448 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:28:28.448 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T23:28:28.521 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T23:28:28.521 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T23:28:28.521 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T23:28:28.521 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T23:28:28.522 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T23:28:28.522 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T23:28:28.522 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T23:28:28.522 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T23:28:28.522 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T23:28:28.522 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T23:28:28.965 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T23:28:28.967 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T23:28:28.967 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-17T23:28:29.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T23:28:29.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T23:28:29.155 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T23:28:29.156 INFO:tasks.ceph:role: osd.5 2024-09-17T23:28:29.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:28:29.156 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-09-17T23:28:29.229 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T23:28:29.230 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T23:28:29.230 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T23:28:29.230 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T23:28:29.714 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-09-17T23:28:29.716 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-09-17T23:28:29.716 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-17T23:28:29.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T23:28:29.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:29.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:29.993+0000 7f2d0fc1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T23:28:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:29.997+0000 7f2d0fc1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T23:28:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:29.997+0000 7f2d0fc1a640 -1 bdev(0x5616b94c9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T23:28:29.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:29.997+0000 7f2d0fc1a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T23:28:33.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T23:28:33.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:33.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:33.281+0000 7fc52d819640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T23:28:33.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:33.282+0000 7fc52d819640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T23:28:33.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:33.282+0000 7fc52d819640 -1 bdev(0x55b5aa7ad800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T23:28:33.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:33.282+0000 7fc52d819640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T23:28:36.187 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T23:28:36.218 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:36.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:36.309+0000 7fbdd15cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T23:28:36.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:36.309+0000 7fbdd15cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T23:28:36.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:36.309+0000 7fbdd15cc640 -1 bdev(0x56368e0db800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T23:28:36.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-17T23:28:36.309+0000 7fbdd15cc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T23:28:39.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T23:28:39.250 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T23:28:39.250 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:39.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T23:28:39.281 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:39.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T23:28:39.352 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:39.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T23:28:39.423 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:39.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T23:28:39.454 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:39.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T23:28:39.525 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:39.526 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T23:28:39.593 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:39.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T23:28:39.660 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-17T23:28:39.660 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T23:28:39.679 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T23:28:39.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T23:28:39.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T23:28:39.708 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-09-17T23:28:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCSEOpmuvmXERAAPWwrds7Rhyo5XyIcE7ph8w== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCVEOpm4uGSExAA2IVEJyCnoxCB86XKtqvMHA== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCYEOpmVUa8JRAAxyH8hVcFXRO+VpGnwxPRHQ== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCPEOpmBC7ZLxAAgrDeNdxqFiWHxffLJjP69w== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCdEOpmeB9DOxAAMqsQlGX+WHQowih0xVKDjQ== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQChEOpmzF3ZEBAAdqAFn+lmjV8z7RPNzRLN8A== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCkEOpmzzh+EhAADvyXE8icfQQV4a1UIr5G8A== 2024-09-17T23:28:39.709 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-17T23:28:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCPEOpmWalgMxAAcRXpak0pgZJcsRT3S8mK/w== 2024-09-17T23:28:39.724 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2024-09-17T23:28:39.724 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCSEOpmuvmXERAAPWwrds7Rhyo5XyIcE7ph8w== 2024-09-17T23:28:39.724 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCVEOpm4uGSExAA2IVEJyCnoxCB86XKtqvMHA== 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCYEOpmVUa8JRAAxyH8hVcFXRO+VpGnwxPRHQ== 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPEOpmBC7ZLxAAgrDeNdxqFiWHxffLJjP69w== 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCdEOpmeB9DOxAAMqsQlGX+WHQowih0xVKDjQ== 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQChEOpmzF3ZEBAAdqAFn+lmjV8z7RPNzRLN8A== 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCkEOpmzzh+EhAADvyXE8icfQQV4a1UIr5G8A== 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-17T23:28:39.725 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPEOpmWalgMxAAcRXpak0pgZJcsRT3S8mK/w== 2024-09-17T23:28:39.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T23:28:39.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T23:28:39.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:39.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:40.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:40.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:40.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:40.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T23:28:40.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T23:28:40.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T23:28:40.203 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T23:28:40.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T23:28:40.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T23:28:40.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T23:28:40.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T23:28:40.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T23:28:40.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T23:28:40.540 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T23:28:40.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T23:28:40.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T23:28:40.692 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:40.695 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T23:28:40.748 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T23:28:40.749 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T23:28:40.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T23:28:40.751 INFO:tasks.ceph.mon.a:Started 2024-09-17T23:28:40.751 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T23:28:40.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T23:28:40.753 INFO:tasks.ceph.mon.c:Started 2024-09-17T23:28:40.753 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T23:28:40.754 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T23:28:40.791 INFO:tasks.ceph.mon.b:Started 2024-09-17T23:28:40.791 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T23:28:40.791 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T23:28:40.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T23:28:40.794 INFO:tasks.ceph.mgr.x:Started 2024-09-17T23:28:40.794 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T23:28:40.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T23:28:46.195 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:46.192+0000 7f455da14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T23:28:46.239 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:46.236+0000 7f455da14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T23:28:46.386 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:46.384+0000 7f455da14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T23:28:46.439 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:46.437+0000 7f455da14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T23:28:46.610 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:46.607+0000 7f455da14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T23:28:46.725 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:46.723+0000 7f455da14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T23:28:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-17T23:28:46.996 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T23:28:46.996 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T23:28:46.997 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:46.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T23:28:47.024 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:47.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T23:28:47.060 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:47.058+0000 7f455da14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T23:28:47.090 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-17T23:28:47.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T23:28:47.116 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:47.114+0000 7f455da14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T23:28:47.157 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:47.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T23:28:47.183 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:47.183 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T23:28:47.249 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-17T23:28:47.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T23:28:47.275 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:47.273+0000 7f455da14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T23:28:47.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd new 2c921380-aff6-4e43-bc78-48a393ed3673 0 2024-09-17T23:28:47.523 INFO:teuthology.orchestra.run.smithi037.stdout:0 2024-09-17T23:28:47.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd new c95ad2cb-7bfe-4d1a-8280-640a59b6e73e 1 2024-09-17T23:28:47.719 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-17T23:28:47.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd new c7d869ca-7f8b-46df-882e-57da88ef6eda 2 2024-09-17T23:28:47.999 INFO:teuthology.orchestra.run.smithi037.stdout:2 2024-09-17T23:28:48.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd new ef4b1c62-ae14-4e5a-9914-d674364f3051 3 2024-09-17T23:28:48.200 INFO:teuthology.orchestra.run.smithi037.stdout:3 2024-09-17T23:28:48.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd new fcd61898-fc45-4d4a-83b2-4f1bf9720309 4 2024-09-17T23:28:48.346 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:48.344+0000 7f455da14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T23:28:48.395 INFO:teuthology.orchestra.run.smithi037.stdout:4 2024-09-17T23:28:48.402 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:48.400+0000 7f455da14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T23:28:48.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd new 1846216e-c369-4918-88be-300c9ee5397f 5 2024-09-17T23:28:48.589 INFO:teuthology.orchestra.run.smithi037.stdout:5 2024-09-17T23:28:48.600 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T23:28:48.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T23:28:48.603 INFO:tasks.ceph.osd.0:Started 2024-09-17T23:28:48.603 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T23:28:48.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T23:28:48.605 INFO:tasks.ceph.osd.1:Started 2024-09-17T23:28:48.606 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T23:28:48.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T23:28:48.608 INFO:tasks.ceph.osd.2:Started 2024-09-17T23:28:48.608 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T23:28:48.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T23:28:48.610 INFO:tasks.ceph.osd.3:Started 2024-09-17T23:28:48.611 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T23:28:48.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T23:28:48.613 INFO:tasks.ceph.osd.4:Started 2024-09-17T23:28:48.613 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T23:28:48.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T23:28:48.616 INFO:tasks.ceph.osd.5:Started 2024-09-17T23:28:48.616 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:28:48.802 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:28:48.803 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:28:48.584872+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T23:28:48.584872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:28:48.810 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:48.808+0000 7f455da14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T23:28:48.813 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T23:28:48.814 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T23:28:48.899 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:48.897+0000 7f455da14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T23:28:49.011 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.009+0000 7f455da14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T23:28:49.060 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.058+0000 7f455da14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T23:28:49.179 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.176+0000 7f455da14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T23:28:49.234 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.232+0000 7f455da14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T23:28:49.281 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.279+0000 7f455da14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T23:28:49.398 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.396+0000 7f455da14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T23:28:49.418 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T23:28:49.451 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.449+0000 7f455da14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T23:28:49.504 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.502+0000 7f455da14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T23:28:49.576 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.574+0000 7f455da14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T23:28:49.585 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-17T23:28:49.585 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":8,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:28:48.584872+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T23:28:48.584872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:28:49.595 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-17T23:28:49.622 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.620+0000 7f455da14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T23:28:49.669 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:49.666+0000 7f455da14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T23:28:49.815 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:28:49.812+0000 7fabe5ee0640 -1 Falling back to public interface 2024-09-17T23:28:49.819 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:28:49.816+0000 7f26d7a13640 -1 Falling back to public interface 2024-09-17T23:28:49.837 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:28:49.835+0000 7f9b8b80d640 -1 Falling back to public interface 2024-09-17T23:28:50.065 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:28:50.063+0000 7f880480c640 -1 Falling back to public interface 2024-09-17T23:28:50.083 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:28:50.081+0000 7f7892444640 -1 Falling back to public interface 2024-09-17T23:28:50.295 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:28:50.292+0000 7fda1c00c640 -1 Falling back to public interface 2024-09-17T23:28:50.353 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:50.351+0000 7f455da14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T23:28:51.367 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:51.364+0000 7f44fd5a2640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T23:28:51.367 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:28:51.365+0000 7f44fd5a2640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T23:28:53.376 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:28:53.373+0000 7f9b8b80d640 -1 osd.3 0 log_to_monitors true 2024-09-17T23:28:53.609 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:28:53.606+0000 7f26d7a13640 -1 osd.1 0 log_to_monitors true 2024-09-17T23:28:53.829 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:28:53.827+0000 7f880480c640 -1 osd.5 0 log_to_monitors true 2024-09-17T23:28:53.831 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:28:53.829+0000 7fabe5ee0640 -1 osd.2 0 log_to_monitors true 2024-09-17T23:28:53.885 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:28:53.883+0000 7f7892444640 -1 osd.4 0 log_to_monitors true 2024-09-17T23:28:54.046 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:28:54.043+0000 7fda1c00c640 -1 osd.0 0 log_to_monitors true 2024-09-17T23:28:56.200 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T23:28:56.455 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:28:56.453+0000 7f788e3d3640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T23:28:56.471 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:28:56.469+0000 7f78861f3640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:28:56.479 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:28:56.477+0000 7f9b8724b640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T23:28:56.480 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:28:56.478+0000 7f88000c9640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T23:28:56.482 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:28:56.479+0000 7fda178c9640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T23:28:56.489 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:28:56.487+0000 7f87f7ee9640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:28:56.490 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:28:56.488+0000 7f9b7f06b640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:28:56.499 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:28:56.496+0000 7fda0f6e9640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:28:56.506 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:28:56.503+0000 7fabe1e6f640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T23:28:56.521 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:28:56.519+0000 7fabd9c8f640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:28:56.554 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:28:56.551+0000 7f26d32f5640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T23:28:56.563 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:28:56.561+0000 7f26cb115640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:28:56.632 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-17T23:28:56.632 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:28:55.373455+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T23:28:48.584872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:28:56.642 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-17T23:29:03.248 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T23:29:03.546 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-17T23:29:03.546 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":14,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:29:00.465201+0000","last_up_change":"2024-09-17T23:28:57.456516+0000","last_in_change":"2024-09-17T23:28:48.584872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T23:28:58.406361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6817","nonce":2793055955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6819","nonce":2793055955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6823","nonce":2793055955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6821","nonce":2793055955}]},"public_addr":"172.21.15.31:6817/2793055955","cluster_addr":"172.21.15.31:6819/2793055955","heartbeat_back_addr":"172.21.15.31:6823/2793055955","heartbeat_front_addr":"172.21.15.31:6821/2793055955","state":["exists","up"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6809","nonce":2784091455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6811","nonce":2784091455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6815","nonce":2784091455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6813","nonce":2784091455}]},"public_addr":"172.21.15.31:6809/2784091455","cluster_addr":"172.21.15.31:6811/2784091455","heartbeat_back_addr":"172.21.15.31:6815/2784091455","heartbeat_front_addr":"172.21.15.31:6813/2784091455","state":["exists","up"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6801","nonce":464841260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6803","nonce":464841260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6807","nonce":464841260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6805","nonce":464841260}]},"public_addr":"172.21.15.31:6801/464841260","cluster_addr":"172.21.15.31:6803/464841260","heartbeat_back_addr":"172.21.15.31:6807/464841260","heartbeat_front_addr":"172.21.15.31:6805/464841260","state":["exists","up"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6801","nonce":1436186932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6803","nonce":1436186932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6807","nonce":1436186932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6805","nonce":1436186932}]},"public_addr":"172.21.15.37:6801/1436186932","cluster_addr":"172.21.15.37:6803/1436186932","heartbeat_back_addr":"172.21.15.37:6807/1436186932","heartbeat_front_addr":"172.21.15.37:6805/1436186932","state":["exists","up"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6817","nonce":154081689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6819","nonce":154081689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6823","nonce":154081689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6821","nonce":154081689}]},"public_addr":"172.21.15.37:6817/154081689","cluster_addr":"172.21.15.37:6819/154081689","heartbeat_back_addr":"172.21.15.37:6823/154081689","heartbeat_front_addr":"172.21.15.37:6821/154081689","state":["exists","up"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6809","nonce":1870185623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6811","nonce":1870185623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6815","nonce":1870185623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6813","nonce":1870185623}]},"public_addr":"172.21.15.37:6809/1870185623","cluster_addr":"172.21.15.37:6811/1870185623","heartbeat_back_addr":"172.21.15.37:6815/1870185623","heartbeat_front_addr":"172.21.15.37:6813/1870185623","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:55.062494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.641591+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.788407+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.360450+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.895512+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.797662+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:29:03.557 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-17T23:29:03.557 INFO:tasks.ceph:Creating RBD pool 2024-09-17T23:29:03.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T23:29:04.491 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-09-17T23:29:04.506 DEBUG:teuthology.orchestra.run.smithi031:> rbd --cluster ceph pool init rbd 2024-09-17T23:29:07.513 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T23:29:07.514 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T23:29:07.514 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T23:29:07.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T23:29:07.515 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T23:29:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:07.875 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":4020979354},{"type":"v1","addr":"172.21.15.37:6825","nonce":4020979354}]},"active_addr":"172.21.15.37:6825/4020979354","active_change":"2024-09-17T23:28:50.354103+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1545028468}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2341924081}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2366472772}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":652344894}]}]} 2024-09-17T23:29:07.878 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T23:29:07.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T23:29:07.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:29:08.169 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:29:07.497277+0000","last_up_change":"2024-09-17T23:28:57.456516+0000","last_in_change":"2024-09-17T23:28:48.584872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T23:28:58.406361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T23:29:03.876212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6817","nonce":2793055955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6819","nonce":2793055955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6823","nonce":2793055955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6821","nonce":2793055955}]},"public_addr":"172.21.15.31:6817/2793055955","cluster_addr":"172.21.15.31:6819/2793055955","heartbeat_back_addr":"172.21.15.31:6823/2793055955","heartbeat_front_addr":"172.21.15.31:6821/2793055955","state":["exists","up"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6809","nonce":2784091455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6811","nonce":2784091455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6815","nonce":2784091455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6813","nonce":2784091455}]},"public_addr":"172.21.15.31:6809/2784091455","cluster_addr":"172.21.15.31:6811/2784091455","heartbeat_back_addr":"172.21.15.31:6815/2784091455","heartbeat_front_addr":"172.21.15.31:6813/2784091455","state":["exists","up"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6801","nonce":464841260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6803","nonce":464841260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6807","nonce":464841260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6805","nonce":464841260}]},"public_addr":"172.21.15.31:6801/464841260","cluster_addr":"172.21.15.31:6803/464841260","heartbeat_back_addr":"172.21.15.31:6807/464841260","heartbeat_front_addr":"172.21.15.31:6805/464841260","state":["exists","up"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6801","nonce":1436186932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6803","nonce":1436186932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6807","nonce":1436186932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6805","nonce":1436186932}]},"public_addr":"172.21.15.37:6801/1436186932","cluster_addr":"172.21.15.37:6803/1436186932","heartbeat_back_addr":"172.21.15.37:6807/1436186932","heartbeat_front_addr":"172.21.15.37:6805/1436186932","state":["exists","up"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6817","nonce":154081689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6819","nonce":154081689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6823","nonce":154081689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6821","nonce":154081689}]},"public_addr":"172.21.15.37:6817/154081689","cluster_addr":"172.21.15.37:6819/154081689","heartbeat_back_addr":"172.21.15.37:6823/154081689","heartbeat_front_addr":"172.21.15.37:6821/154081689","state":["exists","up"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6809","nonce":1870185623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6811","nonce":1870185623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6815","nonce":1870185623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6813","nonce":1870185623}]},"public_addr":"172.21.15.37:6809/1870185623","cluster_addr":"172.21.15.37:6811/1870185623","heartbeat_back_addr":"172.21.15.37:6815/1870185623","heartbeat_front_addr":"172.21.15.37:6813/1870185623","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:55.062494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.641591+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.788407+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.360450+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.895512+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.797662+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:29:08.181 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T23:29:08.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:29:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:08.470 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:29:07.497277+0000","last_up_change":"2024-09-17T23:28:57.456516+0000","last_in_change":"2024-09-17T23:28:48.584872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T23:28:58.406361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T23:29:03.876212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6817","nonce":2793055955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6819","nonce":2793055955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6823","nonce":2793055955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6821","nonce":2793055955}]},"public_addr":"172.21.15.31:6817/2793055955","cluster_addr":"172.21.15.31:6819/2793055955","heartbeat_back_addr":"172.21.15.31:6823/2793055955","heartbeat_front_addr":"172.21.15.31:6821/2793055955","state":["exists","up"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6809","nonce":2784091455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6811","nonce":2784091455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6815","nonce":2784091455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6813","nonce":2784091455}]},"public_addr":"172.21.15.31:6809/2784091455","cluster_addr":"172.21.15.31:6811/2784091455","heartbeat_back_addr":"172.21.15.31:6815/2784091455","heartbeat_front_addr":"172.21.15.31:6813/2784091455","state":["exists","up"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6801","nonce":464841260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6803","nonce":464841260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6807","nonce":464841260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6805","nonce":464841260}]},"public_addr":"172.21.15.31:6801/464841260","cluster_addr":"172.21.15.31:6803/464841260","heartbeat_back_addr":"172.21.15.31:6807/464841260","heartbeat_front_addr":"172.21.15.31:6805/464841260","state":["exists","up"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6801","nonce":1436186932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6803","nonce":1436186932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6807","nonce":1436186932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6805","nonce":1436186932}]},"public_addr":"172.21.15.37:6801/1436186932","cluster_addr":"172.21.15.37:6803/1436186932","heartbeat_back_addr":"172.21.15.37:6807/1436186932","heartbeat_front_addr":"172.21.15.37:6805/1436186932","state":["exists","up"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6817","nonce":154081689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6819","nonce":154081689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6823","nonce":154081689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6821","nonce":154081689}]},"public_addr":"172.21.15.37:6817/154081689","cluster_addr":"172.21.15.37:6819/154081689","heartbeat_back_addr":"172.21.15.37:6823/154081689","heartbeat_front_addr":"172.21.15.37:6821/154081689","state":["exists","up"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6809","nonce":1870185623}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6811","nonce":1870185623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6815","nonce":1870185623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1870185623},{"type":"v1","addr":"172.21.15.37:6813","nonce":1870185623}]},"public_addr":"172.21.15.37:6809/1870185623","cluster_addr":"172.21.15.37:6811/1870185623","heartbeat_back_addr":"172.21.15.37:6815/1870185623","heartbeat_front_addr":"172.21.15.37:6813/1870185623","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:55.062494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.641591+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.788407+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.360450+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.895512+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.797662+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:29:08.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T23:29:08.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T23:29:08.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T23:29:08.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T23:29:08.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T23:29:08.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T23:29:08.642 INFO:teuthology.orchestra.run.smithi031.stdout:47244640262 2024-09-17T23:29:08.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:29:08.658 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:08.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:29:08.687 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:08.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:29:08.703 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:08.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:29:08.710 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:08.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:29:08.713 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:08.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:29:09.066 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-17T23:29:09.070 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-17T23:29:09.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.3 2024-09-17T23:29:09.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.2 2024-09-17T23:29:09.114 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-17T23:29:09.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.1 2024-09-17T23:29:09.169 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-17T23:29:09.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.4 2024-09-17T23:29:09.193 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-17T23:29:09.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.0 2024-09-17T23:29:09.226 INFO:teuthology.orchestra.run.smithi031.stdout:47244640259 2024-09-17T23:29:09.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.5 2024-09-17T23:29:10.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:29:10.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:29:10.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:29:10.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:29:10.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:29:10.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:29:10.475 INFO:teuthology.orchestra.run.smithi031.stdout:47244640260 2024-09-17T23:29:10.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.1 2024-09-17T23:29:10.550 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:10.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2024-09-17T23:29:10.567 DEBUG:teuthology.parallel:result is None 2024-09-17T23:29:10.619 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:10.624 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:10.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.4 2024-09-17T23:29:10.633 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:10.633 DEBUG:teuthology.parallel:result is None 2024-09-17T23:29:10.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-09-17T23:29:10.642 DEBUG:teuthology.parallel:result is None 2024-09-17T23:29:10.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2024-09-17T23:29:10.648 DEBUG:teuthology.parallel:result is None 2024-09-17T23:29:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:47244640261 2024-09-17T23:29:10.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.5 2024-09-17T23:29:10.691 DEBUG:teuthology.parallel:result is None 2024-09-17T23:29:11.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:29:11.797 INFO:teuthology.orchestra.run.smithi031.stdout:47244640262 2024-09-17T23:29:11.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.1 2024-09-17T23:29:11.810 DEBUG:teuthology.parallel:result is None 2024-09-17T23:29:11.810 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T23:29:11.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:29:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:12.096 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:29:12.108 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-17T23:29:10.362265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163232,"kb_used_data":1044,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067808,"statfs":{"total":579820584960,"available":579653435392,"internally_reserved":0,"allocated":1069056,"data_stored":1487080,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9566,"internal_metadata":165927586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.874323"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503624+0000","last_change":"2024-09-17T23:29:05.499485+0000","last_active":"2024-09-17T23:29:05.503624+0000","last_peered":"2024-09-17T23:29:05.503624+0000","last_clean":"2024-09-17T23:29:05.503624+0000","last_became_active":"2024-09-17T23:29:05.498582+0000","last_became_peered":"2024-09-17T23:29:05.498582+0000","last_unstale":"2024-09-17T23:29:05.503624+0000","last_undegraded":"2024-09-17T23:29:05.503624+0000","last_fullsized":"2024-09-17T23:29:05.503624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:22.120133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.501252+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:29:07.501252+0000","last_peered":"2024-09-17T23:29:07.501252+0000","last_clean":"2024-09-17T23:29:07.501252+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:29:07.501252+0000","last_undegraded":"2024-09-17T23:29:07.501252+0000","last_fullsized":"2024-09-17T23:29:07.501252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503972+0000","last_change":"2024-09-17T23:29:05.502177+0000","last_active":"2024-09-17T23:29:05.503972+0000","last_peered":"2024-09-17T23:29:05.503972+0000","last_clean":"2024-09-17T23:29:05.503972+0000","last_became_active":"2024-09-17T23:29:05.501907+0000","last_became_peered":"2024-09-17T23:29:05.501907+0000","last_unstale":"2024-09-17T23:29:05.503972+0000","last_undegraded":"2024-09-17T23:29:05.503972+0000","last_fullsized":"2024-09-17T23:29:05.503972+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:22.672619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.332836+0000","last_change":"2024-09-17T23:29:05.499616+0000","last_active":"2024-09-17T23:29:07.332836+0000","last_peered":"2024-09-17T23:29:07.332836+0000","last_clean":"2024-09-17T23:29:07.332836+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:29:07.332836+0000","last_undegraded":"2024-09-17T23:29:07.332836+0000","last_fullsized":"2024-09-17T23:29:07.332836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:11.833328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.506889+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:29:07.506889+0000","last_peered":"2024-09-17T23:29:07.506889+0000","last_clean":"2024-09-17T23:29:07.506889+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:29:07.506889+0000","last_undegraded":"2024-09-17T23:29:07.506889+0000","last_fullsized":"2024-09-17T23:29:07.506889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.501279+0000","last_change":"2024-09-17T23:29:05.495752+0000","last_active":"2024-09-17T23:29:05.501279+0000","last_peered":"2024-09-17T23:29:05.501279+0000","last_clean":"2024-09-17T23:29:05.501279+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:29:05.501279+0000","last_undegraded":"2024-09-17T23:29:05.501279+0000","last_fullsized":"2024-09-17T23:29:05.501279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:08:54.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.498252+0000","last_change":"2024-09-17T23:29:05.496676+0000","last_active":"2024-09-17T23:29:05.498252+0000","last_peered":"2024-09-17T23:29:05.498252+0000","last_clean":"2024-09-17T23:29:05.498252+0000","last_became_active":"2024-09-17T23:29:05.496146+0000","last_became_peered":"2024-09-17T23:29:05.496146+0000","last_unstale":"2024-09-17T23:29:05.498252+0000","last_undegraded":"2024-09-17T23:29:05.498252+0000","last_fullsized":"2024-09-17T23:29:05.498252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:37.928488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503887+0000","last_change":"2024-09-17T23:29:05.498776+0000","last_active":"2024-09-17T23:29:05.503887+0000","last_peered":"2024-09-17T23:29:05.503887+0000","last_clean":"2024-09-17T23:29:05.503887+0000","last_became_active":"2024-09-17T23:29:05.498403+0000","last_became_peered":"2024-09-17T23:29:05.498403+0000","last_unstale":"2024-09-17T23:29:05.503887+0000","last_undegraded":"2024-09-17T23:29:05.503887+0000","last_fullsized":"2024-09-17T23:29:05.503887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:46.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":166,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.495302+0000","last_change":"2024-09-17T23:28:59.681303+0000","last_active":"2024-09-17T23:29:05.495302+0000","last_peered":"2024-09-17T23:29:05.495302+0000","last_clean":"2024-09-17T23:29:05.495302+0000","last_became_active":"2024-09-17T23:28:59.680283+0000","last_became_peered":"2024-09-17T23:28:59.680283+0000","last_unstale":"2024-09-17T23:29:05.495302+0000","last_undegraded":"2024-09-17T23:29:05.495302+0000","last_fullsized":"2024-09-17T23:29:05.495302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:19.601164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":642205,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":637321,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:29:12.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:29:12.396 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:12.396 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:29:12.408 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-17T23:29:12.362674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163232,"kb_used_data":1044,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067808,"statfs":{"total":579820584960,"available":579653435392,"internally_reserved":0,"allocated":1069056,"data_stored":1487080,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9566,"internal_metadata":165927586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.873221"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503624+0000","last_change":"2024-09-17T23:29:05.499485+0000","last_active":"2024-09-17T23:29:05.503624+0000","last_peered":"2024-09-17T23:29:05.503624+0000","last_clean":"2024-09-17T23:29:05.503624+0000","last_became_active":"2024-09-17T23:29:05.498582+0000","last_became_peered":"2024-09-17T23:29:05.498582+0000","last_unstale":"2024-09-17T23:29:05.503624+0000","last_undegraded":"2024-09-17T23:29:05.503624+0000","last_fullsized":"2024-09-17T23:29:05.503624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:22.120133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.501252+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:29:07.501252+0000","last_peered":"2024-09-17T23:29:07.501252+0000","last_clean":"2024-09-17T23:29:07.501252+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:29:07.501252+0000","last_undegraded":"2024-09-17T23:29:07.501252+0000","last_fullsized":"2024-09-17T23:29:07.501252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503972+0000","last_change":"2024-09-17T23:29:05.502177+0000","last_active":"2024-09-17T23:29:05.503972+0000","last_peered":"2024-09-17T23:29:05.503972+0000","last_clean":"2024-09-17T23:29:05.503972+0000","last_became_active":"2024-09-17T23:29:05.501907+0000","last_became_peered":"2024-09-17T23:29:05.501907+0000","last_unstale":"2024-09-17T23:29:05.503972+0000","last_undegraded":"2024-09-17T23:29:05.503972+0000","last_fullsized":"2024-09-17T23:29:05.503972+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:22.672619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.332836+0000","last_change":"2024-09-17T23:29:05.499616+0000","last_active":"2024-09-17T23:29:07.332836+0000","last_peered":"2024-09-17T23:29:07.332836+0000","last_clean":"2024-09-17T23:29:07.332836+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:29:07.332836+0000","last_undegraded":"2024-09-17T23:29:07.332836+0000","last_fullsized":"2024-09-17T23:29:07.332836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:11.833328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.506889+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:29:07.506889+0000","last_peered":"2024-09-17T23:29:07.506889+0000","last_clean":"2024-09-17T23:29:07.506889+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:29:07.506889+0000","last_undegraded":"2024-09-17T23:29:07.506889+0000","last_fullsized":"2024-09-17T23:29:07.506889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.501279+0000","last_change":"2024-09-17T23:29:05.495752+0000","last_active":"2024-09-17T23:29:05.501279+0000","last_peered":"2024-09-17T23:29:05.501279+0000","last_clean":"2024-09-17T23:29:05.501279+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:29:05.501279+0000","last_undegraded":"2024-09-17T23:29:05.501279+0000","last_fullsized":"2024-09-17T23:29:05.501279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:08:54.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.498252+0000","last_change":"2024-09-17T23:29:05.496676+0000","last_active":"2024-09-17T23:29:05.498252+0000","last_peered":"2024-09-17T23:29:05.498252+0000","last_clean":"2024-09-17T23:29:05.498252+0000","last_became_active":"2024-09-17T23:29:05.496146+0000","last_became_peered":"2024-09-17T23:29:05.496146+0000","last_unstale":"2024-09-17T23:29:05.498252+0000","last_undegraded":"2024-09-17T23:29:05.498252+0000","last_fullsized":"2024-09-17T23:29:05.498252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:37.928488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503887+0000","last_change":"2024-09-17T23:29:05.498776+0000","last_active":"2024-09-17T23:29:05.503887+0000","last_peered":"2024-09-17T23:29:05.503887+0000","last_clean":"2024-09-17T23:29:05.503887+0000","last_became_active":"2024-09-17T23:29:05.498403+0000","last_became_peered":"2024-09-17T23:29:05.498403+0000","last_unstale":"2024-09-17T23:29:05.503887+0000","last_undegraded":"2024-09-17T23:29:05.503887+0000","last_fullsized":"2024-09-17T23:29:05.503887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:46.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":166,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.495302+0000","last_change":"2024-09-17T23:28:59.681303+0000","last_active":"2024-09-17T23:29:05.495302+0000","last_peered":"2024-09-17T23:29:05.495302+0000","last_clean":"2024-09-17T23:29:05.495302+0000","last_became_active":"2024-09-17T23:28:59.680283+0000","last_became_peered":"2024-09-17T23:28:59.680283+0000","last_unstale":"2024-09-17T23:29:05.495302+0000","last_undegraded":"2024-09-17T23:29:05.495302+0000","last_fullsized":"2024-09-17T23:29:05.495302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:19.601164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":642205,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":637321,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:29:12.409 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T23:29:12.409 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T23:29:12.409 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T23:29:12.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T23:29:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:12.737 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T23:29:12.749 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T23:29:12.750 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T23:29:12.760 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T23:29:12.760 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-17T23:29:12.760 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T23:29:12.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T23:29:13.071 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T23:29:13.071 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T23:29:13.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:29:13.359 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:29:13.371 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-17T23:29:12.362674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163232,"kb_used_data":1044,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067808,"statfs":{"total":579820584960,"available":579653435392,"internally_reserved":0,"allocated":1069056,"data_stored":1487080,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9566,"internal_metadata":165927586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.873221"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503624+0000","last_change":"2024-09-17T23:29:05.499485+0000","last_active":"2024-09-17T23:29:05.503624+0000","last_peered":"2024-09-17T23:29:05.503624+0000","last_clean":"2024-09-17T23:29:05.503624+0000","last_became_active":"2024-09-17T23:29:05.498582+0000","last_became_peered":"2024-09-17T23:29:05.498582+0000","last_unstale":"2024-09-17T23:29:05.503624+0000","last_undegraded":"2024-09-17T23:29:05.503624+0000","last_fullsized":"2024-09-17T23:29:05.503624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:22.120133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.501252+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:29:07.501252+0000","last_peered":"2024-09-17T23:29:07.501252+0000","last_clean":"2024-09-17T23:29:07.501252+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:29:07.501252+0000","last_undegraded":"2024-09-17T23:29:07.501252+0000","last_fullsized":"2024-09-17T23:29:07.501252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503972+0000","last_change":"2024-09-17T23:29:05.502177+0000","last_active":"2024-09-17T23:29:05.503972+0000","last_peered":"2024-09-17T23:29:05.503972+0000","last_clean":"2024-09-17T23:29:05.503972+0000","last_became_active":"2024-09-17T23:29:05.501907+0000","last_became_peered":"2024-09-17T23:29:05.501907+0000","last_unstale":"2024-09-17T23:29:05.503972+0000","last_undegraded":"2024-09-17T23:29:05.503972+0000","last_fullsized":"2024-09-17T23:29:05.503972+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:22.672619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.332836+0000","last_change":"2024-09-17T23:29:05.499616+0000","last_active":"2024-09-17T23:29:07.332836+0000","last_peered":"2024-09-17T23:29:07.332836+0000","last_clean":"2024-09-17T23:29:07.332836+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:29:07.332836+0000","last_undegraded":"2024-09-17T23:29:07.332836+0000","last_fullsized":"2024-09-17T23:29:07.332836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:11.833328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.506889+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:29:07.506889+0000","last_peered":"2024-09-17T23:29:07.506889+0000","last_clean":"2024-09-17T23:29:07.506889+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:29:07.506889+0000","last_undegraded":"2024-09-17T23:29:07.506889+0000","last_fullsized":"2024-09-17T23:29:07.506889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.501279+0000","last_change":"2024-09-17T23:29:05.495752+0000","last_active":"2024-09-17T23:29:05.501279+0000","last_peered":"2024-09-17T23:29:05.501279+0000","last_clean":"2024-09-17T23:29:05.501279+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:29:05.501279+0000","last_undegraded":"2024-09-17T23:29:05.501279+0000","last_fullsized":"2024-09-17T23:29:05.501279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:08:54.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.498252+0000","last_change":"2024-09-17T23:29:05.496676+0000","last_active":"2024-09-17T23:29:05.498252+0000","last_peered":"2024-09-17T23:29:05.498252+0000","last_clean":"2024-09-17T23:29:05.498252+0000","last_became_active":"2024-09-17T23:29:05.496146+0000","last_became_peered":"2024-09-17T23:29:05.496146+0000","last_unstale":"2024-09-17T23:29:05.498252+0000","last_undegraded":"2024-09-17T23:29:05.498252+0000","last_fullsized":"2024-09-17T23:29:05.498252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:37.928488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503887+0000","last_change":"2024-09-17T23:29:05.498776+0000","last_active":"2024-09-17T23:29:05.503887+0000","last_peered":"2024-09-17T23:29:05.503887+0000","last_clean":"2024-09-17T23:29:05.503887+0000","last_became_active":"2024-09-17T23:29:05.498403+0000","last_became_peered":"2024-09-17T23:29:05.498403+0000","last_unstale":"2024-09-17T23:29:05.503887+0000","last_undegraded":"2024-09-17T23:29:05.503887+0000","last_fullsized":"2024-09-17T23:29:05.503887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:46.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":166,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.495302+0000","last_change":"2024-09-17T23:28:59.681303+0000","last_active":"2024-09-17T23:29:05.495302+0000","last_peered":"2024-09-17T23:29:05.495302+0000","last_clean":"2024-09-17T23:29:05.495302+0000","last_became_active":"2024-09-17T23:28:59.680283+0000","last_became_peered":"2024-09-17T23:28:59.680283+0000","last_unstale":"2024-09-17T23:29:05.495302+0000","last_undegraded":"2024-09-17T23:29:05.495302+0000","last_fullsized":"2024-09-17T23:29:05.495302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:19.601164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":642205,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":637321,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:29:13.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:29:13.662 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:29:13.674 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-17T23:29:12.362674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163232,"kb_used_data":1044,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067808,"statfs":{"total":579820584960,"available":579653435392,"internally_reserved":0,"allocated":1069056,"data_stored":1487080,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9566,"internal_metadata":165927586},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.873221"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503624+0000","last_change":"2024-09-17T23:29:05.499485+0000","last_active":"2024-09-17T23:29:05.503624+0000","last_peered":"2024-09-17T23:29:05.503624+0000","last_clean":"2024-09-17T23:29:05.503624+0000","last_became_active":"2024-09-17T23:29:05.498582+0000","last_became_peered":"2024-09-17T23:29:05.498582+0000","last_unstale":"2024-09-17T23:29:05.503624+0000","last_undegraded":"2024-09-17T23:29:05.503624+0000","last_fullsized":"2024-09-17T23:29:05.503624+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:22.120133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.501252+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:29:07.501252+0000","last_peered":"2024-09-17T23:29:07.501252+0000","last_clean":"2024-09-17T23:29:07.501252+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:29:07.501252+0000","last_undegraded":"2024-09-17T23:29:07.501252+0000","last_fullsized":"2024-09-17T23:29:07.501252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503972+0000","last_change":"2024-09-17T23:29:05.502177+0000","last_active":"2024-09-17T23:29:05.503972+0000","last_peered":"2024-09-17T23:29:05.503972+0000","last_clean":"2024-09-17T23:29:05.503972+0000","last_became_active":"2024-09-17T23:29:05.501907+0000","last_became_peered":"2024-09-17T23:29:05.501907+0000","last_unstale":"2024-09-17T23:29:05.503972+0000","last_undegraded":"2024-09-17T23:29:05.503972+0000","last_fullsized":"2024-09-17T23:29:05.503972+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:22.672619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.332836+0000","last_change":"2024-09-17T23:29:05.499616+0000","last_active":"2024-09-17T23:29:07.332836+0000","last_peered":"2024-09-17T23:29:07.332836+0000","last_clean":"2024-09-17T23:29:07.332836+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:29:07.332836+0000","last_undegraded":"2024-09-17T23:29:07.332836+0000","last_fullsized":"2024-09-17T23:29:07.332836+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:11.833328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":21,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-17T23:29:07.506889+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:29:07.506889+0000","last_peered":"2024-09-17T23:29:07.506889+0000","last_clean":"2024-09-17T23:29:07.506889+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:29:07.506889+0000","last_undegraded":"2024-09-17T23:29:07.506889+0000","last_fullsized":"2024-09-17T23:29:07.506889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.501279+0000","last_change":"2024-09-17T23:29:05.495752+0000","last_active":"2024-09-17T23:29:05.501279+0000","last_peered":"2024-09-17T23:29:05.501279+0000","last_clean":"2024-09-17T23:29:05.501279+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:29:05.501279+0000","last_undegraded":"2024-09-17T23:29:05.501279+0000","last_fullsized":"2024-09-17T23:29:05.501279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:08:54.558534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.498252+0000","last_change":"2024-09-17T23:29:05.496676+0000","last_active":"2024-09-17T23:29:05.498252+0000","last_peered":"2024-09-17T23:29:05.498252+0000","last_clean":"2024-09-17T23:29:05.498252+0000","last_became_active":"2024-09-17T23:29:05.496146+0000","last_became_peered":"2024-09-17T23:29:05.496146+0000","last_unstale":"2024-09-17T23:29:05.498252+0000","last_undegraded":"2024-09-17T23:29:05.498252+0000","last_fullsized":"2024-09-17T23:29:05.498252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:37.928488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.503887+0000","last_change":"2024-09-17T23:29:05.498776+0000","last_active":"2024-09-17T23:29:05.503887+0000","last_peered":"2024-09-17T23:29:05.503887+0000","last_clean":"2024-09-17T23:29:05.503887+0000","last_became_active":"2024-09-17T23:29:05.498403+0000","last_became_peered":"2024-09-17T23:29:05.498403+0000","last_unstale":"2024-09-17T23:29:05.503887+0000","last_undegraded":"2024-09-17T23:29:05.503887+0000","last_fullsized":"2024-09-17T23:29:05.503887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:46.158921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":166,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-17T23:29:05.495302+0000","last_change":"2024-09-17T23:28:59.681303+0000","last_active":"2024-09-17T23:29:05.495302+0000","last_peered":"2024-09-17T23:29:05.495302+0000","last_clean":"2024-09-17T23:29:05.495302+0000","last_became_active":"2024-09-17T23:28:59.680283+0000","last_became_peered":"2024-09-17T23:28:59.680283+0000","last_unstale":"2024-09-17T23:29:05.495302+0000","last_undegraded":"2024-09-17T23:29:05.495302+0000","last_fullsized":"2024-09-17T23:29:05.495302+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:19.601164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":642205,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":147456,"data_stored":46953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640262,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":204800,"data_stored":637321,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":56824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:29:13.675 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T23:29:13.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 18 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:fsid b662004c-f841-43da-8884-84d1805b0cbb 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-17T23:28:46.056204+0000 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-17T23:29:07.497277+0000 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 4 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-17T23:29:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout: removed_snaps_queue [2~1] 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 6 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/2793055955,v1:172.21.15.31:6817/2793055955] [v2:172.21.15.31:6818/2793055955,v1:172.21.15.31:6819/2793055955] exists,up 2c921380-aff6-4e43-bc78-48a393ed3673 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/2784091455,v1:172.21.15.31:6809/2784091455] [v2:172.21.15.31:6810/2784091455,v1:172.21.15.31:6811/2784091455] exists,up c95ad2cb-7bfe-4d1a-8280-640a59b6e73e 2024-09-17T23:29:13.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/464841260,v1:172.21.15.31:6801/464841260] [v2:172.21.15.31:6802/464841260,v1:172.21.15.31:6803/464841260] exists,up c7d869ca-7f8b-46df-882e-57da88ef6eda 2024-09-17T23:29:13.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6800/1436186932,v1:172.21.15.37:6801/1436186932] [v2:172.21.15.37:6802/1436186932,v1:172.21.15.37:6803/1436186932] exists,up ef4b1c62-ae14-4e5a-9914-d674364f3051 2024-09-17T23:29:13.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6816/154081689,v1:172.21.15.37:6817/154081689] [v2:172.21.15.37:6818/154081689,v1:172.21.15.37:6819/154081689] exists,up fcd61898-fc45-4d4a-83b2-4f1bf9720309 2024-09-17T23:29:13.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6808/1870185623,v1:172.21.15.37:6809/1870185623] [v2:172.21.15.37:6810/1870185623,v1:172.21.15.37:6811/1870185623] exists,up 1846216e-c369-4918-88be-300c9ee5397f 2024-09-17T23:29:13.976 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/2793055955,v1:172.21.15.31:6817/2793055955] [v2:172.21.15.31:6818/2793055955,v1:172.21.15.31:6819/2793055955] exists,up 2c921380-aff6-4e43-bc78-48a393ed3673', 'osd.1 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/2784091455,v1:172.21.15.31:6809/2784091455] [v2:172.21.15.31:6810/2784091455,v1:172.21.15.31:6811/2784091455] exists,up c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/464841260,v1:172.21.15.31:6801/464841260] [v2:172.21.15.31:6802/464841260,v1:172.21.15.31:6803/464841260] exists,up c7d869ca-7f8b-46df-882e-57da88ef6eda', 'osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6800/1436186932,v1:172.21.15.37:6801/1436186932] [v2:172.21.15.37:6802/1436186932,v1:172.21.15.37:6803/1436186932] exists,up ef4b1c62-ae14-4e5a-9914-d674364f3051', 'osd.4 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6816/154081689,v1:172.21.15.37:6817/154081689] [v2:172.21.15.37:6818/154081689,v1:172.21.15.37:6819/154081689] exists,up fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'osd.5 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6808/1870185623,v1:172.21.15.37:6809/1870185623] [v2:172.21.15.37:6810/1870185623,v1:172.21.15.37:6811/1870185623] exists,up 1846216e-c369-4918-88be-300c9ee5397f'] 2024-09-17T23:29:13.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T23:29:14.079 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mon.a", 2024-09-17T23:29:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster": "ceph", 2024-09-17T23:29:14.079 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket_mode": "", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_client_required": "cephx, none", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_cluster_required": "cephx", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_debug": "true", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T23:29:14.080 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_required": "cephx", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_supported": "cephx", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio": "true", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_reap_max": "16", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_async_discard": "false", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_block_size": "4096", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio": "false", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T23:29:14.081 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_enable_discard": "false", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry": "3", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash": "0", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring": "false", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_hipri": "false", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T23:29:14.082 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_type": "", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_allocator": "hybrid", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_buffered_io": "true", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T23:29:14.083 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery": "false", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_sync_write": "false", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocator": "hybrid", 2024-09-17T23:29:14.084 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T23:29:14.085 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_create": "true", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_create": "false", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_path": "", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_size": "0", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_path": "", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_size": "107374182400", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_create": "false", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_path": "", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs": "true", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T23:29:14.086 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune": "true", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size": "0", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T23:29:14.087 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_type": "2q", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_clone_cow": "true", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T23:29:14.088 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_mode": "none", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_freelist": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T23:29:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T23:29:14.090 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fail_eio": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T23:29:14.091 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T23:29:14.092 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size": "0", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T23:29:14.093 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T23:29:14.094 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_mem": "512", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_tracing": "false", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T23:29:14.095 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T23:29:14.096 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_version": "2", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_signatures": "false", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_version": "2", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_signatures": "false", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_version": "2", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_sign_messages": "true", 2024-09-17T23:29:14.097 INFO:teuthology.orchestra.run.smithi031.stdout: "chdir": "", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_acl_type": "", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_asio_thread_count": "2", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_mid": "0.750000", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_size": "16384", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_caps_release_delay": "5", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_check_pool_perm": "true", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_force_sync_read": "false", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_getattr_caps": "false", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_features": "", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_remount": "false", 2024-09-17T23:29:14.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_dirsize_rbytes": "true", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_force_lazyio": "false", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_fs": "", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_release_failure": "false", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_inline_size": "4096", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mds_namespace": "", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_metadata": "", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_gid": "-1", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_timeout": "300", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_uid": "-1", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mountpoint": "/", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_notify_timeout": "10", 2024-09-17T23:29:14.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc": "true", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_objects": "1000", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_size": "209715200", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_permissions": "true", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota": "true", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota_df": "true", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_bytes": "0", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_periods": "4", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_min": "131072", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_reconnect_stale": "false", 2024-09-17T23:29:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_shutdown_timeout": "30", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_snapdir": ".snap", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_tick_interval": "1", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_trace": "", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_faked_inos": "false", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_random_mds": "false", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog": "false", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_port": "12201", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_monitors": "default=true", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog": "false", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T23:29:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_level": "info", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "-", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network": "", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network_interface": "", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_isal": "false", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_level": "5", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zstd_level": "1", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T23:29:14.102 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location": "", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook": "", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook_timeout": "10", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "daemonize": "false", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "daos_pool": "tank", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_alienstore": "0/5", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T23:29:14.103 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok": "1/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok_assert_abort": "false", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_auth": "20/20", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bdev": "1/3", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluefs": "1/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluestore": "1/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_buffer": "0/1", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephsqlite": "0/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_client": "0/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_compressor": "1/5", 2024-09-17T23:29:14.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_context": "0/1", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crush": "1/1", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crypto": "1/5", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cyanstore": "0/5", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_dpdk": "1/5", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_eventtrace": "1/5", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filer": "0/1", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filestore": "1/3", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_finisher": "1/1", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_fuse": "1/5", 2024-09-17T23:29:14.105 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_javaclient": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journal": "1/3", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journaler": "0/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_kstore": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_lockdep": "0/1", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mclock": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_balancer": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_locker": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log": "1/5", 2024-09-17T23:29:14.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_migrator": "1/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_memstore": "1/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgr": "2/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgrc": "1/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mon": "20/20", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_monc": "0/10", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ms": "1/1", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_none": "0/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objclass": "0/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objectcacher": "0/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objecter": "0/1", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_optracker": "0/5", 2024-09-17T23:29:14.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_osd": "1/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_paxos": "20/20", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_perfcounter": "1/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_prioritycache": "1/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rados": "0/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd": "0/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_replay": "0/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_refs": "0/0", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_reserver": "1/1", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw": "1/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_access": "1/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T23:29:14.108 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_flight": "1/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_sync": "1/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rocksdb": "4/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_backref": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cache": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_device": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_epm": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_journal": "0/5", 2024-09-17T23:29:14.109 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_lba": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_odata": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_omap": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_onode": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_t": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_tm": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_striper": "0/1", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_test": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_throttle": "1/1", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_timer": "0/1", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_tp": "0/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_trace": "1/5", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T23:29:14.110 INFO:teuthology.orchestra.run.smithi031.stdout: "device_failure_prediction_mode": "none", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_graylog": "false", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_journald": "false", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_stderr": "true", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_syslog": "false", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event_tracing": "false", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_http_port": "9926", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_prio_limit": "5", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T23:29:14.111 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sort_metrics": "true", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_stats_period": "5", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "fake_statfs_for_testing": "0", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "fatal_signal_handlers": "true", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_purge_ops": "10", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_truncate_ops": "128", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_blackhole": "false", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_snap": "true", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_caller_concurrency": "10", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T23:29:14.112 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_omap_check": "false", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_verify_split": "false", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_dump_file": "", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fadvise": "true", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fail_eio": "true", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_size": "128", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap": "false", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T23:29:14.113 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_inject_stall": "0", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_parallel": "false", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_trailing": "false", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_writeahead": "false", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_kill_at": "0", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T23:29:14.114 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_merge_threshold": "-10", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_odsync_write": "false", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend_path": "", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T23:29:14.115 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_threads": "2", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_punch_hole": "false", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T23:29:14.116 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_ops": "50", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_seek_data_hole": "false", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc": "false", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_splice": "false", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_multiple": "2", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_rand_factor": "20", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_update_to": "1000", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T23:29:14.117 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_xfs_extsize": "false", 2024-09-17T23:29:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_zfs_snap": "false", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fio_dir": "/tmp/fio", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_allow_other": "true", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_big_writes": "true", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_debug": "false", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_default_permissions": "false", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_disable_pagecache": "false", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_max_write": "0", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_multithreaded": "true", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_require_active_mds": "true", 2024-09-17T23:29:14.119 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_set_user_groups": "true", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_move": "true", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_read": "true", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_write": "true", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_target_name": "ceph", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_file": "", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_inject_failure": "0", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_interval": "5", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "host": "smithi031", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T23:29:14.120 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "inject_early_sigterm": "false", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_agent_port": "6799", 2024-09-17T23:29:14.121 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_tracing_enable": "false", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_aio": "true", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_align_min_size": "65536", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_align": "true", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_size": "4096", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_dio": "true", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_discard": "false", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_force_aio": "false", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_ignore_corruption": "false", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_entries": "100", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_replay_from": "0", 2024-09-17T23:29:14.122 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_write_header_frequency": "0", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_zero_on_create": "false", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prefetch_periods": "10", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prezero_periods": "5", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_write_head_interval": "15", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "keyfile": "", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_backend": "rocksdb", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T23:29:14.123 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_bytes": "67108864", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_ops": "512", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_onode_map_size": "1024", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_transaction": "false", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "librados_thread_count": "2", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep": "false", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep_force_backtrace": "false", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "log_coarse_timestamps": "true", 2024-09-17T23:29:14.124 INFO:teuthology.orchestra.run.smithi031.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_flush_on_exit": "false", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_port": "12201", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_new": "1000", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_recent": "10000", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stderr_prefix": "", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": "true", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_graylog": "false", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_journald": "false", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_stderr": "false", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_syslog": "false", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T23:29:14.125 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_action_on_write_error": "1", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_alternate_name_max": "8192", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_asio_thread_count": "2", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_export_pin": "true", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_interval": "10", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max": "-1", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max_until": "-1", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_merge_size": "50", 2024-09-17T23:29:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_mode": "0", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_bits": "3", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_size": "10000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T23:29:14.127 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_mid": "0.700000", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T23:29:14.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_interval": "1", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_auth_pins": "false", 2024-09-17T23:29:14.129 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_frag": "false", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_scatterstat": "false", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_subtrees": "false", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_default_dir_hash": "2", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_defer_session_stale": "true", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_entries": "0", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_prefetch": "true", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T23:29:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_early_reply": "true", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enable_op_tracker": "true", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enforce_unique_name": "true", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T23:29:14.131 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_health_dummy": "false", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_join_fs": "", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_journal_format": "1", 2024-09-17T23:29:14.132 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_create_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_export_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_import_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_link_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_openc_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_rename_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T23:29:14.133 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_events": "-1", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_segments": "128", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_pause": "false", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_segment_size": "0", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_threshold": "128", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T23:29:14.134 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_requests": "100000", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_export_size": "20971520", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_file_recover": "32", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_files": "64", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops": "8192", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_metrics_update_interval": "2", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_per_client": "100", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_numa_node": "-1", 2024-09-17T23:29:14.135 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_duration": "600", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_size": "20", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_log_threshold": "5", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_grace": "15", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_interval": "5", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_caps": "30000", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T23:29:14.136 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_interval": "1.000000", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_gid": "0", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_uid": "0", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T23:29:14.137 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_shutdown_check": "0", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_skip_ino": "0", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_min_uid": "0", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_rstat": "false", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_symlink_recovery": "true", 2024-09-17T23:29:14.138 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_exports": "0", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_fragments": "0", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_tick_interval": "5.000000", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_valgrind_exit": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_backtrace": "1", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_scatter": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_sessions": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool_debug": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_set": "false", 2024-09-17T23:29:14.139 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_size": "65536", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_bytes": "134217728", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_messages": "512", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_disabled_modules": "", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_messages": "128", 2024-09-17T23:29:14.140 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_messages": "128", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_duration": "600", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_size": "20", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_op_log_threshold": "5", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_messages": "8192", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_pool": "true", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T23:29:14.141 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_standby_modules": "true", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_period": "5", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_threshold": "5", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_tick_period": "2", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_delete": "true", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_bytes": "104857600", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T23:29:14.142 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_target_rank": "-1", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file_level": "debug", 2024-09-17T23:29:14.143 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_start": "false", 2024-09-17T23:29:14.144 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_trim": "true", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cpu_threads": "4", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_crit": "5", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_warn": "5", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T23:29:14.145 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_json": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_extra_checks": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T23:29:14.146 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_uptime_grace": "60", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_default_strategy": "1", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_timeout": "5.000000", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_enable_op_tracker": "true", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fake_pool_delete": "false", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_force_quorum_join": "false", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T23:29:14.147 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_log_update_period": "5", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_max_detail": "50", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog": "true", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host": "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.37", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host_override": "", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_initial_members": "", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T23:29:14.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease": "5.000000", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_full_interval": "50", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max": "10000", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max_summary": "50", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_epochs": "500", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_osd": "10000", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T23:29:14.149 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_autotune": "true", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_target": "2147483648", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_digest_period": "5", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T23:29:14.150 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_complaint_time": "30", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_duration": "600", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_size": "20", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_log_threshold": "5", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size": "500", 2024-09-17T23:29:14.151 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-17T23:29:14.152 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_report_timeout": "900", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T23:29:14.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_osds": "4", 2024-09-17T23:29:14.154 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_interval": "86400", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_max_keys": "100", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_timeout": "300", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_session_timeout": "300", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_smart_report_timeout": "5", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T23:29:14.155 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_size": "4", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_debug": "false", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_tick_interval": "5", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T23:29:14.156 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_older_version": "true", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T23:29:14.157 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "monmap": "", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T23:29:14.158 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_tracing_enabled": "false", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_op_threads": "3", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_cm": "false", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_device_name": "", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T23:29:14.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_sl": "3", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_type": "ib", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_reap_threshold": "5", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_before_connect": "false", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv4": "true", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv6": "false", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr1": "true", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr2": "true", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_max": "7568", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_min": "6800", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T23:29:14.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_count": "3", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_delay": "5", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_client": "false", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mds": "false", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mgr": "false", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mon": "false", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_osd": "false", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_mode": "crc secure", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_type": "", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_compress_secure": "false", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T23:29:14.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_data": "true", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_header": "true", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bug": "true", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_old_message": "true", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T23:29:14.162 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hugepages": "", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_lro": "true", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_pmd": "", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_port_id": "0", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_on_send": "false", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T23:29:14.163 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_type": "", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_network_congestion": "0", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_accept_failures": "4", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_backoff": "15.000000", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_mode": "none", 2024-09-17T23:29:14.164 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_min_cost": "65536", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_public_type": "", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_service_mode": "crc secure", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_nodelay": "true", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_type": "async+posix", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "no_config_file": "false", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T23:29:14.165 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_ops": "1024", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_timeout": "10.000000", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_blackhole": "false", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "openssl_engine_opts": "", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_ops": "4", 2024-09-17T23:29:14.166 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_slop": "0.020000", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_asio_thread_count": "2", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_max": "512", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_min": "64", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_peering": "false", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T23:29:14.167 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_beacon_report_interval": "300", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_duration": "30", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blkin_trace_all": "false", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T23:29:14.168 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_update_on_start": "true", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_cap": "256", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_op_priority": "63", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_watch_timeout": "30", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_command_max_records": "256", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_compact_on_start": "false", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_on_start": "true", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T23:29:14.169 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_op_order": "false", 2024-09-17T23:29:14.170 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_shutdown": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_trim_objects": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T23:29:14.171 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_notify_timeout": "30", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_enable_op_tracker": "true", 2024-09-17T23:29:14.172 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_info": "true", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown": "true", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_function_tracing": "false", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_grace": "20", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_interval": "6", 2024-09-17T23:29:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_stale": "600", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_size": "5120", 2024-09-17T23:29:14.174 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_kill_backfill_at": "0", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_cache_size": "50", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_dedup": "true", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max": "40", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_name_len": "100", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_size": "0", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_backfills": "1", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_count": "5", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_period": "600", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_name_len": "2048", 2024-09-17T23:29:14.175 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_size": "134217728", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pgls": "1024", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_cost": "8388608", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_objects": "10", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_scrubs": "3", 2024-09-17T23:29:14.176 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_size": "90", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T23:29:14.177 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_profile": "balanced", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T23:29:14.178 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_base": "805306368", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target": "4294967296", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_autotune": "false", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_recovery_priority": "0", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_interval": "5", 2024-09-17T23:29:14.179 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_cache_shards": "32", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_node": "-1", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objecter_finishers": "1", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore": "bluestore", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_fuse": "false", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_tracing": "false", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_duration": "600", 2024-09-17T23:29:14.180 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_size": "20", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_log_threshold": "5", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards": "0", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T23:29:14.181 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_timeout": "15", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_open_classes_on_start": "true", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_os_flags": "0", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_peering_op_priority": "255", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_priority": "5", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T23:29:14.182 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crimson": "false", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T23:29:14.183 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flags": "0", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_min_size": "0", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_size": "2", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_type": "replicated", 2024-09-17T23:29:14.184 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_cost": "20971520", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active": "0", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T23:29:14.185 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_priority": "3", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_priority": "5", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_repair_during_recovery": "false", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T23:29:14.186 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_max": "25", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_cost": "52428800", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_hour": "0", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T23:29:14.187 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_priority": "5", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_data_digest": "false", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_smart_report_timeout": "5", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T23:29:14.188 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_priority": "5", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_transaction_size": "30", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T23:29:14.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tracing": "false", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_use_stale_snap": "false", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_kill_at": "0", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_max_join_drift": "10", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min": "500", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min_wait": "0.050000", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T23:29:14.190 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max": "500", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_min": "250", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_stash_full_interval": "25", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_max": "500", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_min": "250", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "perf": "true", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "-", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrv": "", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "public_bind_addr": "-", 2024-09-17T23:29:14.191 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network": "", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network_interface": "", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_enabled": "false", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_mon_op_timeout": "0", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_osd_op_timeout": "0", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_tracing": "false", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_atime_update_interval": "60", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T23:29:14.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache": "true", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_policy": "writearound", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_size": "33554432", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_compression_hint": "none", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T23:29:14.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_clone_format": "auto", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_data_pool": "", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_features": "61", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_format": "2", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_map_options": "", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_order": "22", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_pool": "rbd", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_count": "0", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T23:29:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler": "simple", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_order": "24", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_pool": "", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_splay_width": "4", 2024-09-17T23:29:14.195 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T23:29:14.196 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T23:29:14.197 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_threads": "1", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T23:29:14.198 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_plugins": "", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T23:29:14.199 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T23:29:14.200 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_tracing": "false", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_names": "true", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_pool": "true", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_base_url": "", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_log_level": "", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_admin_entry": "admin", 2024-09-17T23:29:14.201 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_backend_store": "rados", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_barbican_url": "", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_beast_enable_async": "true", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_enabled": "true", 2024-09-17T23:29:14.202 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_config_store": "rados", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_content_length_compat": "false", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cross_domain_policy": "", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T23:29:14.203 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T23:29:14.204 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T23:29:14.205 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T23:29:14.206 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_port": "6379", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_window": "30", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T23:29:14.207 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T23:29:14.208 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_name": "", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_s3website_name": "", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T23:29:14.209 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_ops_log": "false", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_static_website": "false", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_usage_log": "false", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_expose_bucket": "false", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_extended_http_attrs": "", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_filter": "none", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T23:29:14.210 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_objs": "32", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_init_timeout": "300", 2024-09-17T23:29:14.211 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password": "", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_project": "", 2024-09-17T23:29:14.212 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_user": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_api_version": "2", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T23:29:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_url": "", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_objs": "32", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_worker": "3", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T23:29:14.214 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_uri": "", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_http_headers": "", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T23:29:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_size": "0", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_listing_results": "1000", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_notify_retries": "10", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T23:29:14.216 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T23:29:14.217 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_control_oids": "8", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_numa_node": "-1", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T23:29:14.218 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_tracing": "false", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_token": "", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_url": "", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_rados": "false", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T23:29:14.219 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T23:29:14.220 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_continue": "true", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_tracing": "false", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm": "", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_id": "", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region": "", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T23:29:14.221 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_request_uri": "", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_resolve_cname": "false", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_run_sync_thread": "true", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T23:29:14.222 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_script_uri": "", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_service_provider_name": "", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_id": "", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_secret": "", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_entry": "sts", 2024-09-17T23:29:14.223 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_key": "sts", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_url": "", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_custom_header": "", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_need_stats": "true", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_tenant_name": "", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url": "", 2024-09-17T23:29:14.224 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_lease_period": "120", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T23:29:14.225 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_thread_pool_size": "512", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_comment": "", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_createby": "", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_encoding": "", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_flag": "false", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_origin": "", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_tracker": "", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T23:29:14.226 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_shards": "32", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_use_opa_authz": "false", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache": "false", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T23:29:14.227 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_unique_email": "true", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_verify_ssl": "true", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone": "", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_id": "", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup": "", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_id": "", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T23:29:14.228 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_block_size": "4096", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T23:29:14.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_partition_filters": "false", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_perf": "false", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "run_dir": "/var/run/ceph", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_block_create": "true", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T23:29:14.230 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_device_size": "53687091200", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_full_integrity_check": "false", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_main_device_type": "SSD", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T23:29:14.231 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_segment_size": "67108864", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "set_keepcaps": "false", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "setgroup": "", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser": "", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser_match_path": "", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "thp": "false", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_default_timeout": "60", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T23:29:14.232 INFO:teuthology.orchestra.run.smithi031.stdout: "throttler_perf_counter": "true" 2024-09-17T23:29:14.233 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:14.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T23:29:14.350 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-17T23:29:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T23:29:14.360 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-17T23:29:14.360 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T23:29:14.369 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-17T23:29:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T23:29:14.380 DEBUG:teuthology.orchestra.run.smithi037:> type ceph-objectstore-tool 2024-09-17T23:29:14.399 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T23:29:14.399 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-09-17T23:29:14.415 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T23:29:14.415 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T23:29:14.425 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T23:29:14.425 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T23:29:14.425 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:14.425 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:14.426 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T23:29:14.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:29:14.426 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T23:29:14.426 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T23:29:14.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:14.427 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T23:29:14.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:14.427 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T23:29:14.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:29:14.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-17T23:29:14.441 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:14.438+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:14.481 INFO:teuthology.orchestra.run.smithi139.stdout:mapped writes DISABLED 2024-09-17T23:29:14.482 INFO:teuthology.orchestra.run.smithi139.stdout:mapped reads DISABLED 2024-09-17T23:29:14.482 INFO:teuthology.orchestra.run.smithi139.stdout:Seed set to 9903 2024-09-17T23:29:14.482 INFO:teuthology.orchestra.run.smithi139.stdout:rbd-nbd mode enabled 2024-09-17T23:29:14.528 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:14.527+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:14.590 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:14.591 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:14.596 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:14.596 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:14.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:14.616 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:14.616 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:14.634 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:14.634 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:14.635 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:14.633+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:14.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:14.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:14.670 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:14.670 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:14.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:14.688 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:14.727 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:14.727 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:14.729 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:14.728+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:14.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:14.772 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 19, 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:29:14.582629+0000", 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:28:57.456516+0000", 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:28:48.584872+0000", 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:29:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:29:14.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:29:14.775 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:29:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:29:14.777 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:29:14.778 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:29:14.779 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:29:14.780 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:29:14.781 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:29:14.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:29:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:29:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:29:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:29:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.25, 2024-09-17T23:29:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.25, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.25, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.25, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:29:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 32769, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,creating", 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:29:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "19", 2024-09-17T23:29:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:29:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:29:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": {}, 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:29:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.1299999952316284, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.1299999952316284, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:29:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:29:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:29:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:29:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:29:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:29:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:29:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 15, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1870185623", 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1870185623", 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1870185623", 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1870185623", 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:29:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 2, 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:14.835 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:14.838 INFO:tasks.thrashosds.thrasher:j is {'epoch': 19, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:29:14.582629+0000', 'last_up_change': '2024-09-17T23:28:57.456516+0000', 'last_in_change': '2024-09-17T23:28:48.584872+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1870185623}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1870185623}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1870185623}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1870185623}]}, 'public_addr': '172.21.15.37:6809/1870185623', 'cluster_addr': '172.21.15.37:6811/1870185623', 'heartbeat_back_addr': '172.21.15.37:6815/1870185623', 'heartbeat_front_addr': '172.21.15.37:6813/1870185623', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:29:14.838 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-17T23:29:14.838 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:14.834+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:14.938 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:14.936+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:14.942 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:14.942 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:14.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:15.039 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:15.036+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:15.058 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.058 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:15.139 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:15.137+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:15.173 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.174 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:15.241 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:15.239+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:15.289 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.290 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:15.342 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:15.339+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:15.442 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:15.440+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.446 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:15.543 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:15.540+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:15.593 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:29:15.613 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.613 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:15.644 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:15.642+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:15.744 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:15.743+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:15.770 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.771 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:15.845 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:15.842+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:15.901 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:15.901 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:15.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:15.951 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:15.948+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:16.048 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:16.046+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:16.058 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.058 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.066 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:16.148 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:16.146+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:16.173 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.173 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:16.250 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:16.248+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:16.329 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.329 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:16.382 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:16.379+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:16.445 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.446 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.451 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:16.449+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:16.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:16.552 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:16.549+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:16.602 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.603 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:16.652 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:16.651+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:16.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:16.753 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:16.752+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:16.758 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.758 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:16.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:16.854 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:16.853+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:16.859 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:16.860 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:16.876 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:16.876 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:16.877 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:16.877 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:16.886 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:16.892 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:16.893 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:16.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:16.909 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:16.927 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:16.927 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:16.955 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:16.952+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:16.995 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:16.995 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:69 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'43, mlcod=13'39)", 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.753333+0000", 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.238994193, 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050282740000000001, 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:16.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753333+0000", 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753333+0000", 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753334+0000", 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5300000000000004e-07 2024-09-17T23:29:16.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753403+0000", 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9919000000000006e-05 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753414+0000", 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0271e-05 2024-09-17T23:29:16.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753419+0000", 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8199999999999996e-06 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:16.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753467+0000", 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8306000000000001e-05 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753480+0000", 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3233e-05 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.758323+0000", 2024-09-17T23:29:17.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048426099999999998 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.758361+0000", 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8562000000000001e-05 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:73 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'47, mlcod=13'43)", 2024-09-17T23:29:17.001 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.760587+0000", 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.231740215999999, 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055840430000000003, 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760587+0000", 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760587+0000", 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760588+0000", 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4200000000000005e-07 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760633+0000", 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5021000000000001e-05 2024-09-17T23:29:17.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760645+0000", 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2095e-05 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760650+0000", 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4149999999999998e-06 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760702+0000", 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1303e-05 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760717+0000", 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4935e-05 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766133+0000", 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054167119999999997 2024-09-17T23:29:17.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766171+0000", 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7719999999999998e-05 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:74 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'48, mlcod=13'43)", 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.761082+0000", 2024-09-17T23:29:17.006 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.231245184999999, 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051069449999999999, 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761082+0000", 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761082+0000", 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761083+0000", 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199999999999999e-07 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761088+0000", 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.062e-06 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761099+0000", 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1569999999999999e-05 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761106+0000", 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.63e-06 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.762999+0000", 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018933380000000001 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.763018+0000", 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8431e-05 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766173+0000", 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031556700000000002 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766189+0000", 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5461999999999999e-05 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:84 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'54, mlcod=13'50)", 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.777856+0000", 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.21447087, 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0052776580000000002, 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777856+0000", 2024-09-17T23:29:17.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777856+0000", 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777857+0000", 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999999999999999e-07 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777914+0000", 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7414999999999998e-05 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777924+0000", 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0067e-05 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777929+0000", 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7419999999999997e-06 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777961+0000", 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1917e-05 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777974+0000", 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2799000000000001e-05 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783091+0000", 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051176040000000004 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783134+0000", 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2493999999999997e-05 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:85 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'55, mlcod=13'50)", 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.213832633999999, 2024-09-17T23:29:17.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046766079999999996, 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:17.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6199999999999997e-07 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778501+0000", 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.276e-06 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778512+0000", 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1399e-05 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778518+0000", 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6840000000000001e-06 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778559+0000", 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1214000000000002e-05 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778572+0000", 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2653999999999999e-05 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783137+0000", 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0045653730000000002 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783171+0000", 2024-09-17T23:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4245999999999997e-05 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:112 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'74, mlcod=13'68)", 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.952526+0000", 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.039800665000001, 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0052955399999999996, 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952526+0000", 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952526+0000", 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952527+0000", 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6599999999999996e-07 2024-09-17T23:29:17.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952588+0000", 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0785999999999997e-05 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952599+0000", 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0563e-05 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952604+0000", 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952637+0000", 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2446000000000001e-05 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952649+0000", 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425999999999999e-05 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957780+0000", 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.005131025 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957822+0000", 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1987999999999998e-05 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.025 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:113 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'75, mlcod=13'68)", 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.039271585000002, 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047959250000000004, 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6100000000000001e-07 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953060+0000", 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8530000000000002e-06 2024-09-17T23:29:17.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953068+0000", 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2109999999999998e-06 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953072+0000", 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1579999999999998e-06 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953112+0000", 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9628000000000001e-05 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953126+0000", 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4515000000000001e-05 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957825+0000", 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046981899999999997 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957851+0000", 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6908999999999999e-05 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:114 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'76, mlcod=13'68)", 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.953532+0000", 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.038795294, 2024-09-17T23:29:17.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0043395630000000003, 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953532+0000", 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953532+0000", 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953533+0000", 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8700000000000005e-07 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953538+0000", 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3399999999999997e-06 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953550+0000", 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1688999999999999e-05 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953555+0000", 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1080000000000001e-06 2024-09-17T23:29:17.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953618+0000", 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3244000000000006e-05 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953634+0000", 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5531000000000001e-05 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957853+0000", 2024-09-17T23:29:17.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042189539999999996 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957871+0000", 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.891e-05 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:130 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'88, mlcod=13'85)", 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:17.035 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.689049239999999, 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055141970000000002, 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.82e-07 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303283+0000", 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303291+0000", 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1550000000000007e-06 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303295+0000", 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1150000000000004e-06 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303326+0000", 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0762999999999997e-05 2024-09-17T23:29:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303336+0000", 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122000000000001e-05 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308768+0000", 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054314560000000003 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308792+0000", 2024-09-17T23:29:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.461e-05 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.303537+0000", 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.688790131000001, 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0083550710000000004, 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.040 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303537+0000", 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303537+0000", 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303538+0000", 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303614+0000", 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6341000000000007e-05 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303621+0000", 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4780000000000002e-06 2024-09-17T23:29:17.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303625+0000", 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5310000000000002e-06 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303654+0000", 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9006999999999999e-05 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303664+0000", 2024-09-17T23:29:17.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9150000000000008e-06 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311862+0000", 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081983720000000006 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311892+0000", 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9858000000000001e-05 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.688282423, 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078664449999999997, 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:17.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8800000000000003e-07 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304049+0000", 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7290000000000002e-06 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304056+0000", 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2609999999999996e-06 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304060+0000", 2024-09-17T23:29:17.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.49e-06 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304091+0000", 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1075000000000003e-05 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304102+0000", 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1523e-05 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311894+0000", 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077915980000000003 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311911+0000", 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7280999999999999e-05 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'85)", 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.687858424000002, 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074569090000000003, 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2e-07 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304473+0000", 2024-09-17T23:29:17.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0559999999999998e-06 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304481+0000", 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5429999999999996e-06 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304484+0000", 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304512+0000", 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7787999999999999e-05 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304524+0000", 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.129e-05 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311912+0000", 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0073882729999999999 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311926+0000", 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3856000000000001e-05 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'85)", 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.687507334999999, 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007118615, 2024-09-17T23:29:17.054 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304871+0000", 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0738000000000001e-05 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304878+0000", 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1409999999999999e-06 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304883+0000", 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9069999999999998e-06 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304913+0000", 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9481000000000001e-05 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304925+0000", 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311926+0000", 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0070011939999999996 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311938+0000", 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2279000000000001e-05 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:135 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'93, mlcod=13'85)", 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.687041107999999, 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0066672509999999999, 2024-09-17T23:29:17.059 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3799999999999998e-07 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305290+0000", 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3500000000000001e-06 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305296+0000", 2024-09-17T23:29:17.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305301+0000", 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0440000000000003e-06 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305329+0000", 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7994e-05 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305341+0000", 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1503e-05 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311940+0000", 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0065988460000000002 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311953+0000", 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3658999999999999e-05 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.063 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:136 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'94, mlcod=13'86)", 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.305685+0000", 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.686641857000001, 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062818350000000004, 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305685+0000", 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305685+0000", 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305686+0000", 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305689+0000", 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.738e-06 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305696+0000", 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7089999999999996e-06 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305699+0000", 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.252e-06 2024-09-17T23:29:17.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305730+0000", 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0941000000000001e-05 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305742+0000", 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.128e-05 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311955+0000", 2024-09-17T23:29:17.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.006213105 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311967+0000", 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2368000000000001e-05 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.686220393999999, 2024-09-17T23:29:17.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058731549999999997, 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1200000000000003e-07 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306157+0000", 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9883999999999999e-05 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306165+0000", 2024-09-17T23:29:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6459999999999993e-06 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306168+0000", 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4800000000000001e-06 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306203+0000", 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4727999999999997e-05 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306216+0000", 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3382e-05 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311968+0000", 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057512730000000003 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311980+0000", 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.225e-05 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.072 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:138 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'96, mlcod=13'86)", 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.685736634000001, 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054006449999999999, 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3900000000000005e-07 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306595+0000", 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0679999999999998e-06 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306603+0000", 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.678e-06 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306606+0000", 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.659e-06 2024-09-17T23:29:17.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306639+0000", 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2270000000000001e-05 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306649+0000", 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0754000000000001e-05 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311980+0000", 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053309890000000004 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311991+0000", 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0688e-05 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-17T23:29:17.077 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.306913+0000", 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.685414082000001, 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050921689999999997, 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306913+0000", 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306913+0000", 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306914+0000", 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0200000000000002e-07 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306920+0000", 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0569999999999998e-06 2024-09-17T23:29:17.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306927+0000", 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8959999999999997e-06 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306930+0000", 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2600000000000001e-06 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306960+0000", 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.012e-05 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306971+0000", 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1489e-05 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311992+0000", 2024-09-17T23:29:17.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050203000000000001 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312005+0000", 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3545e-05 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'86)", 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.307277+0000", 2024-09-17T23:29:17.082 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.685049875000001, 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049717499999999996, 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307277+0000", 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307277+0000", 2024-09-17T23:29:17.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307278+0000", 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8699999999999995e-07 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307317+0000", 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9165000000000002e-05 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307325+0000", 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0020000000000006e-06 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307329+0000", 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5860000000000001e-06 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307360+0000", 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1136000000000001e-05 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307370+0000", 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0553e-05 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312220+0000", 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048497770000000004 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312249+0000", 2024-09-17T23:29:17.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9043999999999998e-05 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'86)", 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 16.684612516000001, 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0045645369999999996, 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:17.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8299999999999997e-07 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307720+0000", 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3750000000000002e-06 2024-09-17T23:29:17.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307728+0000", 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5259999999999996e-06 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307731+0000", 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.433e-06 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308823+0000", 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010911499999999999 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308835+0000", 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2627e-05 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312251+0000", 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0034158320000000002 2024-09-17T23:29:17.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312279+0000", 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8110999999999999e-05 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:17.092 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:17.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:17.095 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:17.092+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:17.195 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:17.193+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:17.202 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:17.202 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:17.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:17.296 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:17.295+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:17.318 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:17.318 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:17.318 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:17.318 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:17.318 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:17.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:17.411 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:17.409+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:17.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:05.507131+0000", 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.965292856, 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000203983, 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507131+0000", 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507131+0000", 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507136+0000", 2024-09-17T23:29:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5220000000000001e-06 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507143+0000", 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7689999999999999e-06 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507152+0000", 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4069999999999999e-06 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507159+0000", 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507224+0000", 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4870999999999996e-05 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507335+0000", 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011158300000000001 2024-09-17T23:29:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16)", 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:05.507477+0000", 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.964946791999999, 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016834200000000001, 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507477+0000", 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507477+0000", 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507478+0000", 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4799999999999997e-07 2024-09-17T23:29:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507483+0000", 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9099999999999996e-06 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507492+0000", 2024-09-17T23:29:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3810000000000005e-06 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507499+0000", 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.241e-06 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507543+0000", 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4753000000000003e-05 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.507646+0000", 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102309 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:06.500715+0000", 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.971709285999999, 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018575569999999999, 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500715+0000", 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500715+0000", 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500717+0000", 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.035e-06 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500735+0000", 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8303000000000001e-05 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500753+0000", 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7532000000000001e-05 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500763+0000", 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8330000000000006e-06 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500817+0000", 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4784999999999998e-05 2024-09-17T23:29:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500940+0000", 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000122803 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.501065+0000", 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012492100000000001 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.501849+0000", 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078371500000000004 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502450+0000", 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060106700000000005 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502514+0000", 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4523000000000005e-05 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502572+0000", 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.804e-05 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4236.0:5 2.2 e17/15 ondisk, result = 0)", 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:06.502344+0000", 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.970079775, 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024015899999999999, 2024-09-17T23:29:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502344+0000", 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502344+0000", 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502346+0000", 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3740000000000001e-06 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502352+0000", 2024-09-17T23:29:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6109999999999999e-06 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502365+0000", 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2763e-05 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502372+0000", 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6969999999999996e-06 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502429+0000", 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6706999999999998e-05 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502441+0000", 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-05 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502585+0000", 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000143607 2024-09-17T23:29:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.9701921789999997, 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046920809999999999, 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502234+0000", 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9400000000000001e-06 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502243+0000", 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4290000000000007e-06 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502260+0000", 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6872e-05 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502272+0000", 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1792e-05 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502599+0000", 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032732099999999999 2024-09-17T23:29:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502674+0000", 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.436e-05 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502795+0000", 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121549 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.504847+0000", 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020513620000000002 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506847+0000", 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020007689999999999 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506887+0000", 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9647000000000002e-05 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506924+0000", 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7039999999999998e-05 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4236.0:7 2.2 e18/15 ondisk, result = 0)", 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:07.506745+0000", 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.9656796310000004, 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018785399999999999, 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506745+0000", 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506745+0000", 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506746+0000", 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.333e-06 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506752+0000", 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3439999999999997e-06 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506766+0000", 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4209999999999999e-05 2024-09-17T23:29:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506773+0000", 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.657e-06 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506835+0000", 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1544000000000006e-05 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506842+0000", 2024-09-17T23:29:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1840000000000002e-06 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506932+0000", 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0581999999999994e-05 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4434.0:10 3.1c e22/19 3:3831950c:::rbd_directory:head v 22'1, mlcod=0'0)", 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:16.848899+0000", 2024-09-17T23:29:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.62352522799999999, 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081099000000000004, 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848899+0000", 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848899+0000", 2024-09-17T23:29:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848899+0000", 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848904+0000", 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2159999999999997e-06 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848915+0000", 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0805000000000001e-05 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848925+0000", 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6800000000000005e-06 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.848986+0000", 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0643999999999997e-05 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.849011+0000", 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5333e-05 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.849668+0000", 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065732300000000002 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:16.849710+0000", 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.163e-05 2024-09-17T23:29:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.510 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:17.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:17.512 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:17.509+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:17.612 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:17.611+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:17.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:29:17.660 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:17.661 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:17.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:17.713 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:17.711+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:17.814 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:17.811+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:17.821 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:17.821 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:17.830 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:17.914 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:17.913+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:17.939 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.464873186999998, 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:17.940 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:17.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:17.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:17.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.776736+0000", 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.1595765, 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079011080000000004, 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 87 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.776736+0000", 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.776736+0000", 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.776737+0000", 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.776772+0000", 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5654000000000001e-05 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.776777+0000", 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1619999999999999e-06 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.776781+0000", 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3160000000000001e-06 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.779392+0000", 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002611005 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.779475+0000", 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3585999999999996e-05 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.779613+0000", 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000138069 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.783688+0000", 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040744609999999997 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.784591+0000", 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00090276699999999996 2024-09-17T23:29:17.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.784617+0000", 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6721e-05 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.784637+0000", 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9993999999999999e-05 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.948571+0000", 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.987741846999999, 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094301019999999992, 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-17T23:29:17.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948571+0000", 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948571+0000", 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948572+0000", 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5600000000000005e-07 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948630+0000", 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8485e-05 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948641+0000", 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0508e-05 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948647+0000", 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.952246+0000", 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035997450000000001 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.952323+0000", 2024-09-17T23:29:17.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6811999999999996e-05 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.952460+0000", 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000136892 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955031+0000", 2024-09-17T23:29:17.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025708440000000001 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.957954+0000", 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029236760000000001 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.957982+0000", 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7724000000000001e-05 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958001+0000", 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8788999999999999e-05 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.948657+0000", 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.987655698000001, 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094943279999999998, 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948657+0000", 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948657+0000", 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948657+0000", 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948662+0000", 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5789999999999997e-06 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948669+0000", 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6959999999999999e-06 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948675+0000", 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1510000000000001e-06 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.952898+0000", 2024-09-17T23:29:17.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004223358 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.952940+0000", 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2123999999999999e-05 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.953009+0000", 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140999999999994e-05 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955033+0000", 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020234129999999999 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958054+0000", 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030215820000000001 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958108+0000", 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3874e-05 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958151+0000", 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3013999999999998e-05 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.948689+0000", 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.987624169, 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009536269, 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948689+0000", 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948689+0000", 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948689+0000", 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948693+0000", 2024-09-17T23:29:17.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8720000000000004e-06 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948701+0000", 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3180000000000007e-06 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948705+0000", 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.411e-06 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.953277+0000", 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045720999999999999 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.953320+0000", 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2957999999999998e-05 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.953432+0000", 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001121 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955037+0000", 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016054789999999999 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958184+0000", 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003146762 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958207+0000", 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3379000000000001e-05 2024-09-17T23:29:17.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958225+0000", 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7390000000000001e-05 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.948717+0000", 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.987596187000001, 2024-09-17T23:29:17.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095721950000000004, 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948717+0000", 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948717+0000", 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948717+0000", 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.53e-07 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948812+0000", 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4911e-05 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948821+0000", 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0510000000000002e-06 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948827+0000", 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5740000000000001e-06 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.953871+0000", 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00504339 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.953951+0000", 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9982999999999996e-05 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.954084+0000", 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013270799999999999 2024-09-17T23:29:17.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955040+0000", 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00095673399999999999 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958246+0000", 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003206235 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958269+0000", 2024-09-17T23:29:17.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2875999999999999e-05 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958289+0000", 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9380000000000001e-05 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.948839+0000", 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.987474071000001, 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095838260000000002, 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948839+0000", 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948839+0000", 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948839+0000", 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948843+0000", 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4709999999999999e-06 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948850+0000", 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0350000000000004e-06 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948855+0000", 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.954543+0000", 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056885599999999996 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.954616+0000", 2024-09-17T23:29:17.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2797000000000001e-05 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.954686+0000", 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9999999999999994e-05 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955493+0000", 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080714700000000003 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958332+0000", 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002838209 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958387+0000", 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5517999999999999e-05 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958422+0000", 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5407999999999997e-05 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.948869+0000", 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.987444142000001, 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096208509999999997, 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-17T23:29:17.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948869+0000", 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948869+0000", 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948869+0000", 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948873+0000", 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8229999999999997e-06 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948880+0000", 2024-09-17T23:29:17.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.781e-06 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.948885+0000", 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955065+0000", 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061798629999999998 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955148+0000", 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3249000000000002e-05 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.955283+0000", 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134797 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.957768+0000", 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002485059 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958447+0000", 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067869200000000005 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958472+0000", 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5103999999999999e-05 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.958489+0000", 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7657999999999999e-05 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.302637+0000", 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633676023, 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095341369999999998, 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302637+0000", 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302637+0000", 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302637+0000", 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000003e-07 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302714+0000", 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7134999999999999e-05 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302723+0000", 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7760000000000003e-06 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302727+0000", 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9940000000000002e-06 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303322+0000", 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059488300000000004 2024-09-17T23:29:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303379+0000", 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7052999999999997e-05 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303466+0000", 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6618000000000003e-05 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:17.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308419+0000", 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049532159999999999 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312123+0000", 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037044280000000001 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312150+0000", 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6276000000000001e-05 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312171+0000", 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1183999999999999e-05 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.302736+0000", 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633577220999999, 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009533807, 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:17.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 132 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302736+0000", 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302736+0000", 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302736+0000", 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302738+0000", 2024-09-17T23:29:18.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.295e-06 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302745+0000", 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3940000000000001e-06 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302748+0000", 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8770000000000003e-06 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303861+0000", 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011120559999999999 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303903+0000", 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2014999999999997e-05 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303972+0000", 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9871999999999998e-05 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308423+0000", 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044505359999999997 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312212+0000", 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037886769999999998 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312243+0000", 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0914000000000003e-05 2024-09-17T23:29:18.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312269+0000", 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6767999999999999e-05 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.302803+0000", 2024-09-17T23:29:18.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633509538999999, 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095355140000000001, 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 133 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302803+0000", 2024-09-17T23:29:18.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302803+0000", 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302804+0000", 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302807+0000", 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9909999999999998e-06 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302815+0000", 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2020000000000004e-06 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302819+0000", 2024-09-17T23:29:18.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8970000000000001e-06 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.304247+0000", 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014278380000000001 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.304299+0000", 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2404000000000002e-05 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.304378+0000", 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8497000000000004e-05 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308432+0000", 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004054233 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312299+0000", 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003866792 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312321+0000", 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2056e-05 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312339+0000", 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8108999999999999e-05 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.302850+0000", 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633462695999999, 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095651769999999994, 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 134 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302850+0000", 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302850+0000", 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302851+0000", 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9299999999999998e-07 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302951+0000", 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302961+0000", 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6099999999999995e-06 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302965+0000", 2024-09-17T23:29:18.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9920000000000001e-06 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.304645+0000", 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001679846 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.304690+0000", 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4901999999999998e-05 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.304762+0000", 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2638999999999994e-05 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308446+0000", 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036835790000000002 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312368+0000", 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039217749999999997 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312395+0000", 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312415+0000", 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.302973+0000", 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633339536000001, 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095143799999999994, 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 135 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302973+0000", 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302973+0000", 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302974+0000", 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:18.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302976+0000", 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6790000000000001e-06 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302982+0000", 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1249999999999998e-06 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.302989+0000", 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7569999999999998e-06 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305094+0000", 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021050679999999999 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305145+0000", 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1192e-05 2024-09-17T23:29:18.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305217+0000", 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1196000000000001e-05 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308456+0000", 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032392269999999999 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312443+0000", 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039868409999999996 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312469+0000", 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6349000000000001e-05 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312488+0000", 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8556e-05 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.303006+0000", 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633307112000001, 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095630669999999997, 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 136 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303006+0000", 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303006+0000", 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303006+0000", 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7399999999999999e-07 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303010+0000", 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8959999999999996e-06 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303014+0000", 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4880000000000001e-06 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303029+0000", 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4258e-05 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305508+0000", 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024793240000000002 2024-09-17T23:29:18.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305555+0000", 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7372e-05 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305637+0000", 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1315000000000003e-05 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308467+0000", 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028304599999999999 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312516+0000", 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040486890000000003 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312543+0000", 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7559000000000002e-05 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312569+0000", 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5332000000000001e-05 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.303037+0000", 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633275644000001, 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096021849999999992, 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303037+0000", 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303037+0000", 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303037+0000", 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303112+0000", 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4751000000000003e-05 2024-09-17T23:29:18.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303120+0000", 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7340000000000008e-06 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303129+0000", 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5450000000000003e-06 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305881+0000", 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027521680000000001 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.305931+0000", 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0278000000000002e-05 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306020+0000", 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9121000000000003e-05 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308888+0000", 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028682429999999999 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312597+0000", 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037087270000000002 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312622+0000", 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5103e-05 2024-09-17T23:29:18.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312639+0000", 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7115000000000001e-05 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.303136+0000", 2024-09-17T23:29:18.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633176956, 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095739210000000009, 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303136+0000", 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303136+0000", 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303136+0000", 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303139+0000", 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.219e-06 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303143+0000", 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.659e-06 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303147+0000", 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.647e-06 2024-09-17T23:29:18.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306404+0000", 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032568559999999998 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306445+0000", 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1952000000000001e-05 2024-09-17T23:29:18.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306513+0000", 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7698000000000001e-05 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308892+0000", 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023786689999999999 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312667+0000", 2024-09-17T23:29:18.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037751320000000001 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312691+0000", 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4250999999999999e-05 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312710+0000", 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8470000000000001e-05 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.303151+0000", 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633161949000002, 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096289329999999992, 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-17T23:29:18.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303151+0000", 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303151+0000", 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303151+0000", 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-07 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303153+0000", 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0719999999999998e-06 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303157+0000", 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6169999999999998e-06 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303159+0000", 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6730000000000001e-06 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306751+0000", 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003591183 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306796+0000", 2024-09-17T23:29:18.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5229999999999999e-05 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.306871+0000", 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5402999999999996e-05 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308894+0000", 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020230339999999999 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312738+0000", 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003843295 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312762+0000", 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4202999999999998e-05 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312780+0000", 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8009999999999999e-05 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.303163+0000", 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633149507999999, 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096900690000000008, 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303163+0000", 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303163+0000", 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303163+0000", 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303216+0000", 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3001999999999998e-05 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303222+0000", 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9880000000000001e-06 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303226+0000", 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1920000000000001e-06 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.307083+0000", 2024-09-17T23:29:18.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038575219999999999 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.307133+0000", 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0024000000000001e-05 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.307206+0000", 2024-09-17T23:29:18.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2558999999999994e-05 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308896+0000", 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001690609 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312807+0000", 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003910836 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312835+0000", 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7929e-05 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312853+0000", 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8170999999999999e-05 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:00.303232+0000", 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.633081125, 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096840769999999993, 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303232+0000", 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303232+0000", 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303232+0000", 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0600000000000001e-07 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303236+0000", 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8229999999999997e-06 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303239+0000", 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6109999999999998e-06 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.303242+0000", 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.65e-06 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.307520+0000", 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042775649999999997 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.307579+0000", 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9005999999999997e-05 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.307655+0000", 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6349999999999999e-05 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.308899+0000", 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012442950000000001 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312874+0000", 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039749 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312898+0000", 2024-09-17T23:29:18.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4020999999999999e-05 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:00.312916+0000", 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.755e-05 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:05.493851+0000", 2024-09-17T23:29:18.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.44246156, 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010074098, 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.493851+0000", 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.493851+0000", 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.493853+0000", 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.142e-06 2024-09-17T23:29:18.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.493863+0000", 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3649999999999993e-06 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.493883+0000", 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-05 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.493897+0000", 2024-09-17T23:29:18.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3301e-05 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.494011+0000", 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011384500000000001 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.494021+0000", 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0135e-05 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.495697+0000", 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00167585 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.495707+0000", 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0400999999999999e-05 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.498971+0000", 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032641409999999999 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499154+0000", 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000183239 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499315+0000", 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000160939 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503597+0000", 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042819970000000001 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503847+0000", 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024936499999999998 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503885+0000", 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8368999999999998e-05 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503925+0000", 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0209000000000001e-05 2024-09-17T23:29:18.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.072 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:18.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:18.074 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:18.071+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:18.173 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:18.172+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:18.222 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:18.222 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:18.223 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:18.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:18.315 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:18.313+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:18.375 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:18.373+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:18.378 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:18.378 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:18.378 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:18.378 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:18.378 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:18.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:18.476 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:18.475+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:18.534 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:18.534 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:18.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:18.534 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:18.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4236.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.033009541, 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003809191, 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499873+0000", 2024-09-17T23:29:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.91e-06 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499897+0000", 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4607999999999999e-05 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499912+0000", 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.435e-05 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499975+0000", 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2942999999999997e-05 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.500010+0000", 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4839000000000003e-05 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503630+0000", 2024-09-17T23:29:18.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003619911 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503672+0000", 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2315999999999999e-05 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:12 3.1 e22/19 3:86f27224:::rbd_header.1152f108f2b3:head v 22'3, mlcod=22'1)", 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:17.019038+0000", 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5138339759999999, 2024-09-17T23:29:18.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000825269, 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019038+0000", 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019038+0000", 2024-09-17T23:29:18.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019040+0000", 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2730000000000001e-06 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019057+0000", 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6500000000000001e-05 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:18.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019086+0000", 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8866000000000001e-05 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019101+0000", 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5489999999999999e-05 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019214+0000", 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011244200000000001 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019256+0000", 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2182999999999999e-05 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019832+0000", 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00057584499999999996 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:18.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019863+0000", 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.167e-05 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:18.544 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:18.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:18.561 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:29:18.593 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:18.590+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:18.668 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:18.669 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:18.669 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:18.669 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:18.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:18.678 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:18.676+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:18.756 INFO:teuthology.orchestra.run.smithi139.stdout:2 write 0xe5cc600 thru 0xe5d31d8 (0x6bd9 bytes) 2024-09-17T23:29:18.756 INFO:teuthology.orchestra.run.smithi139.stdout:3 punch from 0x4b99000 to 0x4ba2400, (0x9400 bytes) 2024-09-17T23:29:18.756 INFO:teuthology.orchestra.run.smithi139.stdout:5 punch from 0x1df8c00 to 0x1e02a00, (0x9e00 bytes) 2024-09-17T23:29:18.756 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xa2c5e00 2024-09-17T23:29:18.760 INFO:teuthology.orchestra.run.smithi139.stdout:6 trunc from 0xe5d3200 to 0xa2c5e00 2024-09-17T23:29:18.778 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:18.777+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:18.785 INFO:teuthology.orchestra.run.smithi139.stdout:10 read 0x3425e00 thru 0x342d4ed (0x76ee bytes) 2024-09-17T23:29:18.785 INFO:teuthology.orchestra.run.smithi139.stdout:12 punch from 0x24b1800 to 0x24c1200, (0xfa00 bytes) 2024-09-17T23:29:18.785 INFO:teuthology.orchestra.run.smithi139.stdout:13 punch from 0x9c27600 to 0x9c2ac00, (0x3600 bytes) 2024-09-17T23:29:18.785 INFO:teuthology.orchestra.run.smithi139.stdout:15 read 0x1233800 thru 0x12363ea (0x2beb bytes) 2024-09-17T23:29:18.789 INFO:teuthology.orchestra.run.smithi139.stdout:16 trunc from 0xa2c5e00 to 0x6333e00 2024-09-17T23:29:18.811 INFO:teuthology.orchestra.run.smithi139.stdout:19 read 0x9e9a00 thru 0x9ed10c (0x370d bytes) 2024-09-17T23:29:18.836 INFO:teuthology.orchestra.run.smithi139.stdout:20 write 0xdf26e00 thru 0xdf3595e (0xeb5f bytes) 2024-09-17T23:29:18.840 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:18.840 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:18.840 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:18.840 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:18.840 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:18.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:18.850 INFO:teuthology.orchestra.run.smithi139.stdout:22 trunc from 0xdf35a00 to 0x20df000 2024-09-17T23:29:18.876 INFO:teuthology.orchestra.run.smithi139.stdout:23 read 0xea9600 thru 0xead78d (0x418e bytes) 2024-09-17T23:29:18.880 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:18.878+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:18.898 INFO:teuthology.orchestra.run.smithi139.stdout:29 write 0x768ac00 thru 0x7698943 (0xdd44 bytes) 2024-09-17T23:29:18.900 INFO:teuthology.orchestra.run.smithi139.stdout:32 trunc from 0x7698a00 to 0x9227000 2024-09-17T23:29:18.920 INFO:teuthology.orchestra.run.smithi139.stdout:33 read 0x3ec2e00 thru 0x3ece7b1 (0xb9b2 bytes) 2024-09-17T23:29:18.920 INFO:teuthology.orchestra.run.smithi139.stdout:34 write 0x28f0800 thru 0x28f9ead (0x96ae bytes) 2024-09-17T23:29:18.920 INFO:teuthology.orchestra.run.smithi139.stdout:35 read 0x902e600 thru 0x9031b1c (0x351d bytes) 2024-09-17T23:29:18.920 INFO:teuthology.orchestra.run.smithi139.stdout:37 write 0x892ee00 thru 0x89305b6 (0x17b7 bytes) 2024-09-17T23:29:18.920 INFO:teuthology.orchestra.run.smithi139.stdout:38 read 0x760aa00 thru 0x761563d (0xac3e bytes) 2024-09-17T23:29:18.921 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xa9a5400 2024-09-17T23:29:18.922 INFO:teuthology.orchestra.run.smithi139.stdout:39 trunc from 0x9227000 to 0xa9a5400 2024-09-17T23:29:18.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:18.945 INFO:teuthology.orchestra.run.smithi139.stdout:40 read 0x5d9da00 thru 0x5da86d2 (0xacd3 bytes) 2024-09-17T23:29:18.945 INFO:teuthology.orchestra.run.smithi139.stdout:41 punch from 0x4e67800 to 0x4e6a600, (0x2e00 bytes) 2024-09-17T23:29:18.945 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xc365e00 2024-09-17T23:29:18.947 INFO:teuthology.orchestra.run.smithi139.stdout:43 trunc from 0xa9a5400 to 0xc365e00 2024-09-17T23:29:18.959 INFO:teuthology.orchestra.run.smithi139.stdout:46 write 0xbb39800 thru 0xbb3e9da (0x51db bytes) 2024-09-17T23:29:18.959 INFO:teuthology.orchestra.run.smithi139.stdout:47 punch from 0x4967400 to 0x496e800, (0x7400 bytes) 2024-09-17T23:29:18.964 INFO:teuthology.orchestra.run.smithi139.stdout:48 read 0x9f53c00 thru 0x9f5d241 (0x9642 bytes) 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:95 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'6, mlcod=23'4)", 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.755006+0000", 2024-09-17T23:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.21423869600000001, 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010668470000000001, 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755006+0000", 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:18.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755006+0000", 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755007+0000", 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.119e-06 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755015+0000", 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9170000000000006e-06 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755030+0000", 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5119999999999999e-05 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755037+0000", 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4969999999999998e-06 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:18.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755093+0000", 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6539999999999997e-05 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755108+0000", 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4851999999999999e-05 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756034+0000", 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000925941 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756073+0000", 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8860999999999999e-05 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:96 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'7, mlcod=23'4)", 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.21351410600000001, 2024-09-17T23:29:18.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028084149999999999, 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000002e-07 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755738+0000", 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9149999999999999e-06 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:18.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755748+0000", 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0091999999999999e-05 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755753+0000", 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7559999999999999e-06 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755796+0000", 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2710000000000003e-05 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755824+0000", 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8118000000000001e-05 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758504+0000", 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026798479999999999 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758539+0000", 2024-09-17T23:29:18.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5281000000000003e-05 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:97 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'8, mlcod=23'4)", 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.756481+0000", 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.21276334799999999, 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046646989999999996, 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.982 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756481+0000", 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756481+0000", 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:18.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756482+0000", 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8899999999999998e-07 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756488+0000", 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9499999999999998e-06 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756502+0000", 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3703000000000001e-05 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756517+0000", 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5349000000000002e-05 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756571+0000", 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3346000000000001e-05 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756586+0000", 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5075999999999999e-05 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761110+0000", 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0045241830000000002 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761146+0000", 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6303000000000001e-05 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.986 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:102 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'9, mlcod=23'7)", 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.766468+0000", 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.20277652299999999, 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088604000000000005, 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766468+0000", 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766468+0000", 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766469+0000", 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2299999999999999e-07 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766476+0000", 2024-09-17T23:29:18.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766488+0000", 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2771e-05 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766496+0000", 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1019999999999997e-06 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766562+0000", 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5611999999999995e-05 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.766586+0000", 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3734999999999999e-05 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.767295+0000", 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00070963299999999999 2024-09-17T23:29:18.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.767354+0000", 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8938999999999999e-05 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:109 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'10, mlcod=23'8)", 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.768969+0000", 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.20027561099999999, 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0015039750000000001, 2024-09-17T23:29:18.991 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.768969+0000", 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.768969+0000", 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.768970+0000", 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1600000000000001e-07 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.768975+0000", 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.843e-06 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:18.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.768984+0000", 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5300000000000002e-06 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.768989+0000", 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.758e-06 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.769027+0000", 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7780000000000001e-05 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.769047+0000", 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0064999999999998e-05 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.770442+0000", 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013947300000000001 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.770473+0000", 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1553000000000001e-05 2024-09-17T23:29:18.995 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:126 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'12, mlcod=23'10)", 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.784496+0000", 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.184748831, 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012310349999999999, 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:18.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784496+0000", 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784496+0000", 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784497+0000", 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1349999999999999e-06 2024-09-17T23:29:18.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784506+0000", 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784522+0000", 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5299999999999999e-05 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784530+0000", 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784590+0000", 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9598000000000001e-05 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.784613+0000", 2024-09-17T23:29:18.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3461000000000001e-05 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785680+0000", 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010670580000000001 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785727+0000", 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6758000000000002e-05 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:127 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'13, mlcod=23'10)", 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.785257+0000", 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.18398745, 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00094591100000000002, 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785257+0000", 2024-09-17T23:29:19.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785257+0000", 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785258+0000", 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3099999999999997e-07 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785264+0000", 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2559999999999999e-06 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785273+0000", 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1600000000000004e-06 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785278+0000", 2024-09-17T23:29:19.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9930000000000003e-06 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785327+0000", 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8435e-05 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.785348+0000", 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0968999999999999e-05 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.786169+0000", 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00082071399999999995 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.786203+0000", 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4752999999999997e-05 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:128 3.b 3:d3468d19:::rbd_data.1152f108f2b3.0000000000000009:head [set-alloc-hint object_size 4194304 write_size 4194304,write 727040~64000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.786848+0000", 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.182397158, 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021004370000000001, 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 128 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.786848+0000", 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.786848+0000", 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.786849+0000", 2024-09-17T23:29:19.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.331e-06 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.786990+0000", 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141175 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.787000+0000", 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8369999999999992e-06 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.787008+0000", 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5510000000000002e-06 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.787078+0000", 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0493000000000004e-05 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.787230+0000", 2024-09-17T23:29:19.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000151495 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.787388+0000", 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000158474 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.788690+0000", 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013016600000000001 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.788891+0000", 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000201306 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.788921+0000", 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.987e-05 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.788948+0000", 2024-09-17T23:29:19.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7245000000000001e-05 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:131 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'14, mlcod=23'12)", 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.791901+0000", 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.17734383200000001, 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00084061699999999999, 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.012 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791901+0000", 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791901+0000", 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791902+0000", 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2500000000000005e-07 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791908+0000", 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.657e-06 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791920+0000", 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1533e-05 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791925+0000", 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.592e-06 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.791980+0000", 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4493999999999997e-05 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.792004+0000", 2024-09-17T23:29:19.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3844000000000001e-05 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.792705+0000", 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00070098800000000005 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.792742+0000", 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6783999999999999e-05 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:171 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'18, mlcod=23'16)", 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.850119+0000", 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.119126076, 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00086925100000000003, 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850119+0000", 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850119+0000", 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850120+0000", 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.328e-06 2024-09-17T23:29:19.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850132+0000", 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1989e-05 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850150+0000", 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8101000000000002e-05 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850159+0000", 2024-09-17T23:29:19.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.85e-06 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850231+0000", 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2350999999999999e-05 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850258+0000", 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6427e-05 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850939+0000", 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00068145300000000005 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.850988+0000", 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8751999999999997e-05 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:175 3.b 3:d3468d19:::rbd_data.1152f108f2b3.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.856906+0000", 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.112339262, 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038253990000000002, 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 175 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856906+0000", 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856906+0000", 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856907+0000", 2024-09-17T23:29:19.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8080000000000001e-06 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856914+0000", 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7580000000000003e-06 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856931+0000", 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7269999999999999e-05 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856942+0000", 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0563e-05 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.857011+0000", 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8638000000000003e-05 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.857100+0000", 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9215999999999994e-05 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.857213+0000", 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112848 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.859415+0000", 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022018680000000001 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.860620+0000", 2024-09-17T23:29:19.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001205734 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.860681+0000", 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0362999999999998e-05 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.860731+0000", 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0333000000000001e-05 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:180 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'20, mlcod=23'18)", 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.861499+0000", 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.107745697, 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00077568899999999998, 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861499+0000", 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861499+0000", 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861500+0000", 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1699999999999997e-07 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861506+0000", 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861515+0000", 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6099999999999995e-06 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861521+0000", 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8649999999999998e-06 2024-09-17T23:29:19.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861576+0000", 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4929999999999998e-05 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.861599+0000", 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2659000000000001e-05 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.862245+0000", 2024-09-17T23:29:19.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00064592700000000002 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.862275+0000", 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0236e-05 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:199 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'22, mlcod=23'20)", 2024-09-17T23:29:19.032 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.885361+0000", 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.083884146000000007, 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000778892, 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885361+0000", 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885361+0000", 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885362+0000", 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.017e-06 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885368+0000", 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2219999999999999e-06 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885378+0000", 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0492999999999999e-05 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885384+0000", 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7590000000000003e-06 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885442+0000", 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8236000000000003e-05 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.885466+0000", 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3988000000000001e-05 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.886097+0000", 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00063040600000000002 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.886140+0000", 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2771000000000001e-05 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:215 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'23, mlcod=23'21)", 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.900483+0000", 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.068761483999999998, 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00086694099999999998, 2024-09-17T23:29:19.037 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900483+0000", 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900483+0000", 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900484+0000", 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3900000000000003e-07 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900492+0000", 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2350000000000002e-06 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900505+0000", 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3390000000000001e-05 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900511+0000", 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900569+0000", 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7797999999999998e-05 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.900593+0000", 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3798000000000002e-05 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.901305+0000", 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00071213400000000001 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.901350+0000", 2024-09-17T23:29:19.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5318000000000002e-05 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:218 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'24, mlcod=23'22)", 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.907422+0000", 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.061823116999999997, 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00078204200000000005, 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.042 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907422+0000", 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907422+0000", 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907423+0000", 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.16e-07 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907428+0000", 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4539999999999998e-06 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907438+0000", 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0456e-05 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907445+0000", 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7360000000000004e-06 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907498+0000", 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2429000000000002e-05 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.907522+0000", 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4864000000000001e-05 2024-09-17T23:29:19.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.908163+0000", 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00064096800000000001 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.908204+0000", 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0318999999999998e-05 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:235 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'25, mlcod=23'23)", 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.922196+0000", 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.047048582999999998, 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033276920000000001, 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922196+0000", 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922196+0000", 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922197+0000", 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7000000000000003e-07 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922204+0000", 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9789999999999996e-06 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922218+0000", 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3868e-05 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922224+0000", 2024-09-17T23:29:19.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8409999999999996e-06 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922284+0000", 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0526000000000003e-05 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922310+0000", 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5176000000000001e-05 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925485+0000", 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031757769999999999 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925524+0000", 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8554999999999999e-05 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:236 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'26, mlcod=23'23)", 2024-09-17T23:29:19.051 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.046330126999999999, 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029570210000000002, 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922920+0000", 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6739999999999996e-06 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922928+0000", 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8269999999999998e-06 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922932+0000", 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4490000000000001e-06 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922989+0000", 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7226999999999999e-05 2024-09-17T23:29:19.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.923011+0000", 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1140000000000001e-05 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925842+0000", 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028316679999999999 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925872+0000", 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9493e-05 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:257 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'28, mlcod=23'26)", 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.944758+0000", 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.024487228999999999, 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00083841099999999995, 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944758+0000", 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944758+0000", 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944758+0000", 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4900000000000005e-07 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944767+0000", 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4440000000000008e-06 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944780+0000", 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2677000000000001e-05 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944785+0000", 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8479999999999997e-06 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944853+0000", 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7831999999999997e-05 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.944878+0000", 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4950999999999999e-05 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.945542+0000", 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00066340400000000003 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.945596+0000", 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4406000000000001e-05 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:276 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'30, mlcod=23'28)", 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.959531+0000", 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0097136099999999993, 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00082271199999999997, 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959531+0000", 2024-09-17T23:29:19.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959531+0000", 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959532+0000", 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7300000000000005e-07 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959537+0000", 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3539999999999999e-06 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959548+0000", 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0482999999999999e-05 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959553+0000", 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0259999999999998e-06 2024-09-17T23:29:19.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959601+0000", 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7695000000000003e-05 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.959622+0000", 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1821e-05 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.960307+0000", 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00068420700000000004 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.960354+0000", 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7352999999999999e-05 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.065 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:278 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'31, mlcod=23'29)", 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.963464+0000", 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0057812189999999998, 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00074168499999999996, 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963464+0000", 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963464+0000", 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963464+0000", 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963469+0000", 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7509999999999999e-06 2024-09-17T23:29:19.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963479+0000", 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8919999999999995e-06 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963486+0000", 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9970000000000001e-06 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:19.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963535+0000", 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9159000000000001e-05 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.963552+0000", 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6752000000000002e-05 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.964158+0000", 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00060648099999999997 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:19.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.964205+0000", 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6953000000000003e-05 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:19.070 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:19.071 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:19.072 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:19.072 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:19.069+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:19.073 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:19.074 INFO:teuthology.orchestra.run.smithi139.stdout:50 write 0xc68aa00 thru 0xc690731 (0x5d32 bytes) 2024-09-17T23:29:19.074 INFO:teuthology.orchestra.run.smithi139.stdout:51 read 0x7004a00 thru 0x7010954 (0xbf55 bytes) 2024-09-17T23:29:19.074 INFO:teuthology.orchestra.run.smithi139.stdout:52 read 0x9310800 thru 0x931a30f (0x9b10 bytes) 2024-09-17T23:29:19.074 INFO:teuthology.orchestra.run.smithi139.stdout:53 punch from 0x5e3a800 to 0x5e3e400, (0x3c00 bytes) 2024-09-17T23:29:19.074 INFO:teuthology.orchestra.run.smithi139.stdout:55 trunc from 0xc690800 to 0x9210600 2024-09-17T23:29:19.074 INFO:teuthology.orchestra.run.smithi139.stdout:57 write 0xa40a400 thru 0xa411ebd (0x7abe bytes) 2024-09-17T23:29:19.075 INFO:teuthology.orchestra.run.smithi139.stdout:58 punch from 0x2855200 to 0x2856800, (0x1600 bytes) 2024-09-17T23:29:19.075 INFO:teuthology.orchestra.run.smithi139.stdout:59 write 0xdf8c600 thru 0xdf8d860 (0x1261 bytes) 2024-09-17T23:29:19.075 INFO:teuthology.orchestra.run.smithi139.stdout:60 trunc from 0xdf8da00 to 0x440e200 2024-09-17T23:29:19.075 INFO:teuthology.orchestra.run.smithi139.stdout:64 trunc from 0x440e200 to 0xc0c0c00 2024-09-17T23:29:19.075 INFO:teuthology.orchestra.run.smithi139.stdout:66 punch from 0x78be00 to 0x79a200, (0xe400 bytes) 2024-09-17T23:29:19.076 INFO:teuthology.orchestra.run.smithi139.stdout:67 read 0x2085c00 thru 0x208b819 (0x5c1a bytes) 2024-09-17T23:29:19.076 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xdb73800 2024-09-17T23:29:19.078 INFO:teuthology.orchestra.run.smithi139.stdout:69 trunc from 0xc0c0c00 to 0xdb73800 2024-09-17T23:29:19.082 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:19.089 INFO:teuthology.orchestra.run.smithi139.stdout:70 read 0x9c7f000 thru 0x9c81c62 (0x2c63 bytes) 2024-09-17T23:29:19.090 INFO:teuthology.orchestra.run.smithi139.stdout:71 read 0x3cc4600 thru 0x3cce938 (0xa339 bytes) 2024-09-17T23:29:19.091 INFO:teuthology.orchestra.run.smithi139.stdout:72 write 0xa32c000 thru 0xa32f39c (0x339d bytes) 2024-09-17T23:29:19.091 INFO:teuthology.orchestra.run.smithi139.stdout:73 write 0x6289200 thru 0x62957e4 (0xc5e5 bytes) 2024-09-17T23:29:19.091 INFO:teuthology.orchestra.run.smithi139.stdout:76 punch from 0xd86c000 to 0xd86f400, (0x3400 bytes) 2024-09-17T23:29:19.091 INFO:teuthology.orchestra.run.smithi139.stdout:78 read 0xc328e00 thru 0xc330d9e (0x7f9f bytes) 2024-09-17T23:29:19.099 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:19.099 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:19.110 INFO:teuthology.orchestra.run.smithi139.stdout:79 write 0xeca8600 thru 0xecaa07c (0x1a7d bytes) 2024-09-17T23:29:19.111 INFO:teuthology.orchestra.run.smithi139.stdout:80 read 0x629e00 thru 0x633c0a (0x9e0b bytes) 2024-09-17T23:29:19.112 INFO:teuthology.orchestra.run.smithi139.stdout:84 punch from 0xc919c00 to 0xc91a200, (0x600 bytes) 2024-09-17T23:29:19.112 INFO:teuthology.orchestra.run.smithi139.stdout:87 write 0x5f2ca00 thru 0x5f35383 (0x8984 bytes) 2024-09-17T23:29:19.112 INFO:teuthology.orchestra.run.smithi139.stdout:90 write 0x43dcc00 thru 0x43ea1ed (0xd5ee bytes) 2024-09-17T23:29:19.113 INFO:teuthology.orchestra.run.smithi139.stdout:91 write 0x839c600 thru 0x83a6e68 (0xa869 bytes) 2024-09-17T23:29:19.113 INFO:teuthology.orchestra.run.smithi139.stdout:92 write 0x69e0400 thru 0x69e679b (0x639c bytes) 2024-09-17T23:29:19.113 INFO:teuthology.orchestra.run.smithi139.stdout:95 read 0x3dffa00 thru 0x3e098b0 (0x9eb1 bytes) 2024-09-17T23:29:19.113 INFO:teuthology.orchestra.run.smithi139.stdout:96 write 0x81cb600 thru 0x81d5a35 (0xa436 bytes) 2024-09-17T23:29:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:97 write 0x448a000 thru 0x448ed6e (0x4d6f bytes) 2024-09-17T23:29:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:98 write 0x54d3c00 thru 0x54da541 (0x6942 bytes) 2024-09-17T23:29:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:100 write 0x2404800 thru 0x24054e1 (0xce2 bytes) 2024-09-17T23:29:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:102 read 0x8fac600 thru 0x8fb266b (0x606c bytes) 2024-09-17T23:29:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:103 punch from 0xac21e00 to 0xac25400, (0x3600 bytes) 2024-09-17T23:29:19.114 INFO:teuthology.orchestra.run.smithi139.stdout:105 read 0xa9d4200 thru 0xa9deb31 (0xa932 bytes) 2024-09-17T23:29:19.115 INFO:teuthology.orchestra.run.smithi139.stdout:106 read 0x8d06c00 thru 0x8d1513b (0xe53c bytes) 2024-09-17T23:29:19.115 INFO:teuthology.orchestra.run.smithi139.stdout:108 read 0x98c4000 thru 0x98caa5f (0x6a60 bytes) 2024-09-17T23:29:19.115 INFO:teuthology.orchestra.run.smithi139.stdout:110 write 0xd905a00 thru 0xd907bf7 (0x21f8 bytes) 2024-09-17T23:29:19.117 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:19.122 INFO:teuthology.orchestra.run.smithi139.stdout:112 punch from 0x9575400 to 0x9577600, (0x2200 bytes) 2024-09-17T23:29:19.123 INFO:teuthology.orchestra.run.smithi139.stdout:116 read 0x9e6fa00 thru 0x9e7df21 (0xe522 bytes) 2024-09-17T23:29:19.123 INFO:teuthology.orchestra.run.smithi139.stdout:118 read 0x8848600 thru 0x88506f1 (0x80f2 bytes) 2024-09-17T23:29:19.123 INFO:teuthology.orchestra.run.smithi139.stdout:121 write 0x3d9a000 thru 0x3d9ea9c (0x4a9d bytes) 2024-09-17T23:29:19.124 INFO:teuthology.orchestra.run.smithi139.stdout:122 punch from 0x5527e00 to 0x5534e00, (0xd000 bytes) 2024-09-17T23:29:19.124 INFO:teuthology.orchestra.run.smithi139.stdout:124 read 0x1779e00 thru 0x177b23e (0x143f bytes) 2024-09-17T23:29:19.124 INFO:teuthology.orchestra.run.smithi139.stdout:125 read 0x2a3600 thru 0x2b0985 (0xd386 bytes) 2024-09-17T23:29:19.124 INFO:teuthology.orchestra.run.smithi139.stdout:128 read 0xab78200 thru 0xab7f10b (0x6f0c bytes) 2024-09-17T23:29:19.124 INFO:teuthology.orchestra.run.smithi139.stdout:131 read 0x2dfb200 thru 0x2e0aea3 (0xfca4 bytes) 2024-09-17T23:29:19.124 INFO:teuthology.orchestra.run.smithi139.stdout:132 punch from 0x87e5200 to 0x87eea00, (0x9800 bytes) 2024-09-17T23:29:19.137 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:19.137 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:19.140 INFO:teuthology.orchestra.run.smithi139.stdout:133 trunc from 0xecaa200 to 0xee0600 2024-09-17T23:29:19.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:19.156 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:19.172 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:19.169+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:19.183 INFO:teuthology.orchestra.run.smithi139.stdout:134 write 0x8336e00 thru 0x8345557 (0xe758 bytes) 2024-09-17T23:29:19.184 INFO:teuthology.orchestra.run.smithi139.stdout:135 write 0x222d800 thru 0x2235f47 (0x8748 bytes) 2024-09-17T23:29:19.188 INFO:teuthology.orchestra.run.smithi139.stdout:136 trunc from 0x8345600 to 0xbf2cc00 2024-09-17T23:29:19.198 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:19.198 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:19.207 INFO:teuthology.orchestra.run.smithi139.stdout:137 read 0x73db400 thru 0x73e7dcb (0xc9cc bytes) 2024-09-17T23:29:19.207 INFO:teuthology.orchestra.run.smithi139.stdout:138 read 0x7536000 thru 0x7543d47 (0xdd48 bytes) 2024-09-17T23:29:19.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:19.225 INFO:teuthology.orchestra.run.smithi139.stdout:140 write 0xed0a600 thru 0xed0adb9 (0x7ba bytes) 2024-09-17T23:29:19.226 INFO:teuthology.orchestra.run.smithi139.stdout:141 write 0xb7d0600 thru 0xb7d361a (0x301b bytes) 2024-09-17T23:29:19.242 INFO:teuthology.orchestra.run.smithi139.stdout:143 trunc from 0xed0ae00 to 0x1a59400 2024-09-17T23:29:19.265 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xe5cfe00 2024-09-17T23:29:19.273 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:19.271+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:19.281 INFO:teuthology.orchestra.run.smithi139.stdout:146 trunc from 0x1a59400 to 0xe5cfe00 2024-09-17T23:29:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:148 write 0xa34ce00 thru 0xa359ff5 (0xd1f6 bytes) 2024-09-17T23:29:19.292 INFO:teuthology.orchestra.run.smithi139.stdout:149 punch from 0x2213000 to 0x221d400, (0xa400 bytes) 2024-09-17T23:29:19.293 INFO:teuthology.orchestra.run.smithi139.stdout:151 write 0x1471400 thru 0x1475790 (0x4391 bytes) 2024-09-17T23:29:19.306 INFO:teuthology.orchestra.run.smithi139.stdout:154 trunc from 0xe5cfe00 to 0x37fb000 2024-09-17T23:29:19.333 INFO:teuthology.orchestra.run.smithi139.stdout:155 trunc from 0x37fb000 to 0xb398600 2024-09-17T23:29:19.344 INFO:teuthology.orchestra.run.smithi139.stdout:158 punch from 0x15c1200 to 0x15c8400, (0x7200 bytes) 2024-09-17T23:29:19.346 INFO:teuthology.orchestra.run.smithi139.stdout:161 read 0x3c2200 thru 0x3c8428 (0x6229 bytes) 2024-09-17T23:29:19.346 INFO:teuthology.orchestra.run.smithi139.stdout:163 punch from 0x918fe00 to 0x9194800, (0x4a00 bytes) 2024-09-17T23:29:19.361 INFO:teuthology.orchestra.run.smithi139.stdout:166 write 0xdcc6c00 thru 0xdcd205c (0xb45d bytes) 2024-09-17T23:29:19.361 INFO:teuthology.orchestra.run.smithi139.stdout:167 read 0x125bc00 thru 0x126454d (0x894e bytes) 2024-09-17T23:29:19.361 INFO:teuthology.orchestra.run.smithi139.stdout:168 read 0x967e200 thru 0x967eb52 (0x953 bytes) 2024-09-17T23:29:19.361 INFO:teuthology.orchestra.run.smithi139.stdout:truncating to largest ever: 0xee32600 2024-09-17T23:29:19.361 INFO:teuthology.orchestra.run.smithi139.stdout:172 trunc from 0xdcd2200 to 0xee32600 2024-09-17T23:29:19.363 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:19.363 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:19.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:19.375 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:19.372+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:19.375 INFO:teuthology.orchestra.run.smithi139.stdout:176 write 0xe076400 thru 0xe082db4 (0xc9b5 bytes) 2024-09-17T23:29:19.376 INFO:teuthology.orchestra.run.smithi139.stdout:177 write 0x66bd200 thru 0x66be4f2 (0x12f3 bytes) 2024-09-17T23:29:19.376 INFO:teuthology.orchestra.run.smithi139.stdout:178 punch from 0x32df600 to 0x32e3200, (0x3c00 bytes) 2024-09-17T23:29:19.377 INFO:teuthology.orchestra.run.smithi139.stdout:179 write 0xdf66c00 thru 0xdf71b9f (0xafa0 bytes) 2024-09-17T23:29:19.378 INFO:teuthology.orchestra.run.smithi139.stdout:181 write 0xe447c00 thru 0xe452e29 (0xb22a bytes) 2024-09-17T23:29:19.378 INFO:teuthology.orchestra.run.smithi139.stdout:182 write 0x567aa00 thru 0x5685dfe (0xb3ff bytes) 2024-09-17T23:29:19.378 INFO:teuthology.orchestra.run.smithi139.stdout:184 read 0x9a72800 thru 0x9a80cc4 (0xe4c5 bytes) 2024-09-17T23:29:19.387 INFO:teuthology.orchestra.run.smithi139.stdout:185 trunc from 0xee32600 to 0x78b9e00 2024-09-17T23:29:19.425 INFO:teuthology.orchestra.run.smithi139.stdout:187 write 0xb661e00 thru 0xb670026 (0xe227 bytes) 2024-09-17T23:29:19.426 INFO:teuthology.orchestra.run.smithi139.stdout:189 read 0x2ee9a00 thru 0x2ef4e4f (0xb450 bytes) 2024-09-17T23:29:19.426 INFO:teuthology.orchestra.run.smithi139.stdout:191 read 0x1ceaa00 thru 0x1cef763 (0x4d64 bytes) 2024-09-17T23:29:19.426 INFO:teuthology.orchestra.run.smithi139.stdout:192 write 0xa7da00 thru 0xa813c5 (0x39c6 bytes) 2024-09-17T23:29:19.426 INFO:teuthology.orchestra.run.smithi139.stdout:193 punch from 0x830d800 to 0x8310400, (0x2c00 bytes) 2024-09-17T23:29:19.426 INFO:teuthology.orchestra.run.smithi139.stdout:194 read 0x114e800 thru 0x11587ca (0x9fcb bytes) 2024-09-17T23:29:19.426 INFO:teuthology.orchestra.run.smithi139.stdout:195 read 0x5739600 thru 0x57486a0 (0xf0a1 bytes) 2024-09-17T23:29:19.427 INFO:teuthology.orchestra.run.smithi139.stdout:196 write 0x7afe00 thru 0x7b87a7 (0x89a8 bytes) 2024-09-17T23:29:19.427 INFO:teuthology.orchestra.run.smithi139.stdout:197 read 0x8f49400 thru 0x8f521a8 (0x8da9 bytes) 2024-09-17T23:29:19.427 INFO:teuthology.orchestra.run.smithi139.stdout:198 punch from 0x8823200 to 0x882ec00, (0xba00 bytes) 2024-09-17T23:29:19.427 INFO:teuthology.orchestra.run.smithi139.stdout:200 read 0x8865a00 thru 0x8870829 (0xae2a bytes) 2024-09-17T23:29:19.440 INFO:teuthology.orchestra.run.smithi139.stdout:201 trunc from 0xb670200 to 0x118aa00 2024-09-17T23:29:19.472 INFO:teuthology.orchestra.run.smithi139.stdout:202 trunc from 0x118aa00 to 0xd404e00 2024-09-17T23:29:19.475 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:19.472+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:19.483 INFO:teuthology.orchestra.run.smithi139.stdout:205 write 0xcdd0a00 thru 0xcddcc9e (0xc29f bytes) 2024-09-17T23:29:19.484 INFO:teuthology.orchestra.run.smithi139.stdout:206 read 0x6de1e00 thru 0x6de585f (0x3a60 bytes) 2024-09-17T23:29:19.484 INFO:teuthology.orchestra.run.smithi139.stdout:208 write 0x1412600 thru 0x141b780 (0x9181 bytes) 2024-09-17T23:29:19.492 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:19.492 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:19.494 INFO:teuthology.orchestra.run.smithi139.stdout:209 trunc from 0xd404e00 to 0x5586e00 2024-09-17T23:29:19.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:19.514 INFO:teuthology.orchestra.run.smithi139.stdout:210 write 0x428a000 thru 0x42995d3 (0xf5d4 bytes) 2024-09-17T23:29:19.515 INFO:teuthology.orchestra.run.smithi139.stdout:211 write 0xea0800 thru 0xea866c (0x7e6d bytes) 2024-09-17T23:29:19.516 INFO:teuthology.orchestra.run.smithi139.stdout:213 read 0x337c00 thru 0x341ae2 (0x9ee3 bytes) 2024-09-17T23:29:19.516 INFO:teuthology.orchestra.run.smithi139.stdout:214 read 0x357d400 thru 0x357e644 (0x1245 bytes) 2024-09-17T23:29:19.516 INFO:teuthology.orchestra.run.smithi139.stdout:215 write 0x1420800 thru 0x14292c3 (0x8ac4 bytes) 2024-09-17T23:29:19.531 INFO:teuthology.orchestra.run.smithi139.stdout:216 write 0x6a99800 thru 0x6aa2015 (0x8816 bytes) 2024-09-17T23:29:19.532 INFO:teuthology.orchestra.run.smithi139.stdout:219 read 0x5ab4a00 thru 0x5ac266c (0xdc6d bytes) 2024-09-17T23:29:19.540 INFO:teuthology.orchestra.run.smithi139.stdout:220 trunc from 0x6aa2200 to 0xcbcf000 2024-09-17T23:29:19.554 INFO:teuthology.orchestra.run.smithi139.stdout:221 read 0x1e41a00 thru 0x1e4f9aa (0xdfab bytes) 2024-09-17T23:29:19.555 INFO:teuthology.orchestra.run.smithi139.stdout:222 read 0x1a00e00 thru 0x1a0ede6 (0xdfe7 bytes) 2024-09-17T23:29:19.555 INFO:teuthology.orchestra.run.smithi139.stdout:225 read 0x1619a00 thru 0x161c300 (0x2901 bytes) 2024-09-17T23:29:19.556 INFO:teuthology.orchestra.run.smithi139.stdout:226 punch from 0x4b65a00 to 0x4b72a00, (0xd000 bytes) 2024-09-17T23:29:19.556 INFO:teuthology.orchestra.run.smithi139.stdout:229 read 0x2753400 thru 0x27568db (0x34dc bytes) 2024-09-17T23:29:19.556 INFO:teuthology.orchestra.run.smithi139.stdout:230 write 0xb9e4400 thru 0xb9f3a70 (0xf671 bytes) 2024-09-17T23:29:19.556 INFO:teuthology.orchestra.run.smithi139.stdout:231 read 0xa32d400 thru 0xa33427d (0x6e7e bytes) 2024-09-17T23:29:19.556 INFO:teuthology.orchestra.run.smithi139.stdout:233 write 0x9f8000 thru 0xa012fc (0x92fd bytes) 2024-09-17T23:29:19.557 INFO:teuthology.orchestra.run.smithi139.stdout:234 write 0x1e0000 thru 0x1e3639 (0x363a bytes) 2024-09-17T23:29:19.557 INFO:teuthology.orchestra.run.smithi139.stdout:235 read 0xb386a00 thru 0xb39393b (0xcf3c bytes) 2024-09-17T23:29:19.557 INFO:teuthology.orchestra.run.smithi139.stdout:237 write 0xb39f800 thru 0xb3a0c9a (0x149b bytes) 2024-09-17T23:29:19.568 INFO:teuthology.orchestra.run.smithi139.stdout:238 trunc from 0xcbcf000 to 0x3828800 2024-09-17T23:29:19.575 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:19.573+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:19.592 INFO:teuthology.orchestra.run.smithi139.stdout:239 read 0x3cce00 thru 0x3d8661 (0xb862 bytes) 2024-09-17T23:29:19.606 INFO:teuthology.orchestra.run.smithi139.stdout:241 write 0x6b81000 thru 0x6b8f61f (0xe620 bytes) 2024-09-17T23:29:19.624 INFO:teuthology.orchestra.run.smithi139.stdout:246 write 0xb2eae00 thru 0xb2f72ce (0xc4cf bytes) 2024-09-17T23:29:19.625 INFO:teuthology.orchestra.run.smithi139.stdout:248 read 0x2c56e00 thru 0x2c61ad5 (0xacd6 bytes) 2024-09-17T23:29:19.625 INFO:teuthology.orchestra.run.smithi139.stdout:250 write 0x50f4600 thru 0x50f9f24 (0x5925 bytes) 2024-09-17T23:29:19.625 INFO:teuthology.orchestra.run.smithi139.stdout:253 punch from 0x359dc00 to 0x35a7e00, (0xa200 bytes) 2024-09-17T23:29:19.625 INFO:teuthology.orchestra.run.smithi139.stdout:255 read 0x49cd800 thru 0x49d05be (0x2dbf bytes) 2024-09-17T23:29:19.625 INFO:teuthology.orchestra.run.smithi139.stdout:257 read 0x5379000 thru 0x53880d1 (0xf0d2 bytes) 2024-09-17T23:29:19.626 INFO:teuthology.orchestra.run.smithi139.stdout:258 read 0xa65000 thru 0xa69709 (0x470a bytes) 2024-09-17T23:29:19.626 INFO:teuthology.orchestra.run.smithi139.stdout:259 read 0x33ec00 thru 0x342751 (0x3b52 bytes) 2024-09-17T23:29:19.626 INFO:teuthology.orchestra.run.smithi139.stdout:260 read 0x837ec00 thru 0x838131d (0x271e bytes) 2024-09-17T23:29:19.642 INFO:teuthology.orchestra.run.smithi139.stdout:267 write 0xd6cd800 thru 0xd6d28e9 (0x50ea bytes) 2024-09-17T23:29:19.644 INFO:teuthology.orchestra.run.smithi139.stdout:268 write 0x7af7e00 thru 0x7afd2d7 (0x54d8 bytes) 2024-09-17T23:29:19.644 INFO:teuthology.orchestra.run.smithi139.stdout:270 read 0xa512600 thru 0xa51ac5a (0x865b bytes) 2024-09-17T23:29:19.644 INFO:teuthology.orchestra.run.smithi139.stdout:271 punch from 0xdc3e00 to 0xdcf800, (0xba00 bytes) 2024-09-17T23:29:19.644 INFO:teuthology.orchestra.run.smithi139.stdout:272 clone 1 order 25 su 131072 sc 8 2024-09-17T23:29:19.670 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:19.670 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:19.676 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:19.673+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:19.679 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:19.777 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:19.776+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:19.826 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:19.826 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:19.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:19.839 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:19.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:19.839 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:19.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T23:29:19.883 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:19.881+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:19.942 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:19.942 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:19.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:19.979 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:19.976+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:20.059 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.060 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:20.113 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:20.111+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:20.140 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:29:20.138+0000 7f21c94c5640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2024-09-17T23:29:20.176 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.176 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.180 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:20.179+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:20.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:20.281 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:20.279+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:20.331 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.332 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:20.382 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:20.380+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:20.483 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:20.480+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:20.488 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.488 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:20.562 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.2. 2024-09-17T23:29:20.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:29:20.610 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.610 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.610 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:20.608+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:20.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:20.685 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:20.683+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:20.769 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.769 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:20.787 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:20.784+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:20.887 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:20.884+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:20.898 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:20.898 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:20.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:20.989 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:20.987+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:21.015 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:21.015 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:21.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:21.108 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:21.106+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:21.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:21.171 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:21.171 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:21.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:21.213 INFO:teuthology.orchestra.run.smithi139.stdout:275 write 0x2762800 thru 0x2766482 (0x3c83 bytes) 2024-09-17T23:29:21.213 INFO:teuthology.orchestra.run.smithi139.stdout:276 write 0x2551e00 thru 0x25591da (0x73db bytes) 2024-09-17T23:29:21.213 INFO:teuthology.orchestra.run.smithi139.stdout:278 write 0x16b200 thru 0x170ae5 (0x58e6 bytes) 2024-09-17T23:29:21.213 INFO:teuthology.orchestra.run.smithi139.stdout:284 read 0x7f6400 thru 0x7fbe34 (0x5a35 bytes) 2024-09-17T23:29:21.221 INFO:teuthology.orchestra.run.smithi139.stdout:286 trunc from 0xd6d2a00 to 0x7471600 2024-09-17T23:29:21.222 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:21.220+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:21.285 INFO:teuthology.orchestra.run.smithi139.stdout:291 read 0x419e000 thru 0x41a316e (0x516f bytes) 2024-09-17T23:29:21.287 INFO:teuthology.orchestra.run.smithi139.stdout:292 punch from 0x549e800 to 0x54a6000, (0x7800 bytes) 2024-09-17T23:29:21.291 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:21.288+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:21.305 INFO:teuthology.orchestra.run.smithi139.stdout:293 write 0xcfdf400 thru 0xcfe56b7 (0x62b8 bytes) 2024-09-17T23:29:21.306 INFO:teuthology.orchestra.run.smithi139.stdout:294 write 0xbef2400 thru 0xbef63d6 (0x3fd7 bytes) 2024-09-17T23:29:21.307 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:21.307 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:21.307 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:21.308 INFO:teuthology.orchestra.run.smithi139.stdout:296 write 0x8d63400 thru 0x8d7202e (0xec2f bytes) 2024-09-17T23:29:21.308 INFO:teuthology.orchestra.run.smithi139.stdout:297 write 0x6cbb200 thru 0x6cc5d04 (0xab05 bytes) 2024-09-17T23:29:21.308 INFO:teuthology.orchestra.run.smithi139.stdout:299 write 0x625c000 thru 0x626397d (0x797e bytes) 2024-09-17T23:29:21.308 INFO:teuthology.orchestra.run.smithi139.stdout:305 punch from 0x2462400 to 0x246f800, (0xd400 bytes) 2024-09-17T23:29:21.309 INFO:teuthology.orchestra.run.smithi139.stdout:306 read 0x189b400 thru 0x18a8a29 (0xd62a bytes) 2024-09-17T23:29:21.313 INFO:teuthology.orchestra.run.smithi139.stdout:307 read 0x7ac2200 thru 0x7ac7195 (0x4f96 bytes) 2024-09-17T23:29:21.316 INFO:teuthology.orchestra.run.smithi139.stdout:311 read 0x91ea200 thru 0x91f4a2b (0xa82c bytes) 2024-09-17T23:29:21.316 INFO:teuthology.orchestra.run.smithi139.stdout:314 read 0xce0ae00 thru 0xce15799 (0xa99a bytes) 2024-09-17T23:29:21.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:21.325 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:21.325 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:21.330 INFO:teuthology.orchestra.run.smithi139.stdout:319 trunc from 0xcfe5800 to 0x15b3400 2024-09-17T23:29:21.341 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:21.355 INFO:teuthology.orchestra.run.smithi139.stdout:320 flatten 2024-09-17T23:29:21.359 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:21.359 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:21.377 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:21.380 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:21.384 INFO:teuthology.orchestra.run.smithi139.stdout:321 write 0x2f5ca00 thru 0x2f65cde (0x92df bytes) 2024-09-17T23:29:21.384 INFO:teuthology.orchestra.run.smithi139.stdout:322 trunc from 0x2f65e00 to 0x3085c00 2024-09-17T23:29:21.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:21.394 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:21.401 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:21.398+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:21.417 INFO:teuthology.orchestra.run.smithi139.stdout:325 write 0x4a49200 thru 0x4a58b01 (0xf902 bytes) 2024-09-17T23:29:21.417 INFO:teuthology.orchestra.run.smithi139.stdout:329 write 0x1dfb800 thru 0x1dfbc5f (0x460 bytes) 2024-09-17T23:29:21.417 INFO:teuthology.orchestra.run.smithi139.stdout:332 write 0x2950200 thru 0x295f431 (0xf232 bytes) 2024-09-17T23:29:21.418 INFO:teuthology.orchestra.run.smithi139.stdout:333 read 0x353c800 thru 0x354933b (0xcb3c bytes) 2024-09-17T23:29:21.418 INFO:teuthology.orchestra.run.smithi139.stdout:339 read 0x3d1e000 thru 0x3d23686 (0x5687 bytes) 2024-09-17T23:29:21.444 INFO:teuthology.orchestra.run.smithi139.stdout:340 write 0x7378400 thru 0x7386c2f (0xe830 bytes) 2024-09-17T23:29:21.472 INFO:teuthology.orchestra.run.smithi139.stdout:341 write 0xebe7a00 thru 0xebf72b8 (0xf8b9 bytes) 2024-09-17T23:29:21.472 INFO:teuthology.orchestra.run.smithi139.stdout:342 write 0xdf7b400 thru 0xdf8852a (0xd12b bytes) 2024-09-17T23:29:21.472 INFO:teuthology.orchestra.run.smithi139.stdout:345 write 0x52d8e00 thru 0x52db98e (0x2b8f bytes) 2024-09-17T23:29:21.473 INFO:teuthology.orchestra.run.smithi139.stdout:346 clone 2 order 24 su 524288 sc 8 2024-09-17T23:29:21.481 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:21.481 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:21.481 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:21.481 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:21.481 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:21.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:21.533 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:21.530+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:21.594 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:21.591+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:96 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'7, mlcod=23'4)", 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.882188813, 2024-09-17T23:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028084149999999999, 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755731+0000", 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000002e-07 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755738+0000", 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9149999999999999e-06 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755748+0000", 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0091999999999999e-05 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755753+0000", 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7559999999999999e-06 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755796+0000", 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2710000000000003e-05 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755824+0000", 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8118000000000001e-05 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758504+0000", 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026798479999999999 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758539+0000", 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5281000000000003e-05 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:97 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'8, mlcod=23'4)", 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.756481+0000", 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.8814380550000003, 2024-09-17T23:29:21.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046646989999999996, 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756481+0000", 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756481+0000", 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756482+0000", 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8899999999999998e-07 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756488+0000", 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9499999999999998e-06 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756502+0000", 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3703000000000001e-05 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756517+0000", 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5349000000000002e-05 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756571+0000", 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3346000000000001e-05 2024-09-17T23:29:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756586+0000", 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5075999999999999e-05 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761110+0000", 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0045241830000000002 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761146+0000", 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6303000000000001e-05 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:175 3.b 3:d3468d19:::rbd_data.1152f108f2b3.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.856906+0000", 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.781013969, 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038253990000000002, 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 175 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856906+0000", 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856906+0000", 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856907+0000", 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8080000000000001e-06 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856914+0000", 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7580000000000003e-06 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856931+0000", 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7269999999999999e-05 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.856942+0000", 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0563e-05 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.857011+0000", 2024-09-17T23:29:21.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8638000000000003e-05 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.857100+0000", 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9215999999999994e-05 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.857213+0000", 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112848 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:21.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.859415+0000", 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022018680000000001 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.860620+0000", 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001205734 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.860681+0000", 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0362999999999998e-05 2024-09-17T23:29:21.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.860731+0000", 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0333000000000001e-05 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:235 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'25, mlcod=23'23)", 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.922196+0000", 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.7157232900000001, 2024-09-17T23:29:21.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033276920000000001, 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922196+0000", 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922196+0000", 2024-09-17T23:29:21.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922197+0000", 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7000000000000003e-07 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922204+0000", 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9789999999999996e-06 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922218+0000", 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3868e-05 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922224+0000", 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8409999999999996e-06 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922284+0000", 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0526000000000003e-05 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922310+0000", 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5176000000000001e-05 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925485+0000", 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031757769999999999 2024-09-17T23:29:21.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925524+0000", 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8554999999999999e-05 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:236 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'26, mlcod=23'23)", 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.7150048340000001, 2024-09-17T23:29:21.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029570210000000002, 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:21.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922915+0000", 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922920+0000", 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6739999999999996e-06 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922928+0000", 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8269999999999998e-06 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922932+0000", 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4490000000000001e-06 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922989+0000", 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7226999999999999e-05 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.923011+0000", 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1140000000000001e-05 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925842+0000", 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028316679999999999 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925872+0000", 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9493e-05 2024-09-17T23:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:298 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'34, mlcod=23'31)", 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.976051+0000", 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.661868997, 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002903344, 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976051+0000", 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976051+0000", 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976051+0000", 2024-09-17T23:29:21.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5000000000000002e-07 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976058+0000", 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-06 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976068+0000", 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9860000000000006e-06 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976074+0000", 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5050000000000004e-06 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976131+0000", 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6518000000000003e-05 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976150+0000", 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8909000000000001e-05 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.978914+0000", 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027648199999999999 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.978954+0000", 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9415999999999997e-05 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.671 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:299 3.7 e23/19 3:e64865bb:::rbd_data.1152f108f2b3.0000000000000031:head v 23'1, mlcod=0'0)", 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.977218+0000", 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.6607012029999999, 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044298660000000002, 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977218+0000", 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977218+0000", 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977219+0000", 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0699999999999997e-07 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977236+0000", 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7322000000000001e-05 2024-09-17T23:29:21.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977245+0000", 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0829999999999993e-06 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977252+0000", 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2500000000000003e-06 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977307+0000", 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5544000000000002e-05 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977327+0000", 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0109999999999999e-05 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.981606+0000", 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042785499999999999 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.981648+0000", 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2400000000000001e-05 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:311 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'35, mlcod=23'33)", 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.987130+0000", 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.650789719, 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0030493590000000002, 2024-09-17T23:29:21.676 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987130+0000", 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987130+0000", 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987130+0000", 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5899999999999996e-07 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987135+0000", 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7339999999999999e-06 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987145+0000", 2024-09-17T23:29:21.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.55e-06 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987151+0000", 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7279999999999998e-06 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987195+0000", 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3115999999999999e-05 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.987208+0000", 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3679000000000001e-05 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.990152+0000", 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029434259999999999 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.990179+0000", 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7467e-05 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.680 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:446 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'49, mlcod=23'46)", 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.091222+0000", 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.5466973720000001, 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026397819999999998, 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091222+0000", 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091222+0000", 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091223+0000", 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999996e-07 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091228+0000", 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5400000000000003e-06 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091238+0000", 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6409999999999997e-06 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091244+0000", 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1789999999999996e-06 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091286+0000", 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2347e-05 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091301+0000", 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4377e-05 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.093817+0000", 2024-09-17T23:29:21.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025161900000000002 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.093862+0000", 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4904000000000002e-05 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:447 3.6 e23/19 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head v 23'50, mlcod=23'46)", 2024-09-17T23:29:21.685 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.091816+0000", 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.5461034160000002, 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048075940000000001, 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091816+0000", 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091816+0000", 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091817+0000", 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3300000000000001e-07 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091822+0000", 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3929999999999999e-06 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091834+0000", 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.206e-05 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091841+0000", 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5810000000000002e-06 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091884+0000", 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3414999999999997e-05 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091903+0000", 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8876999999999999e-05 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.096591+0000", 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046874910000000002 2024-09-17T23:29:21.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.096624+0000", 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3043999999999997e-05 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4447.0:451 3.6 e23/19 3:641496d6:::rbd_data.1152f108f2b3.0000000000000036:head v 23'51, mlcod=23'49)", 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.097250+0000", 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.5406695510000001, 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021056600000000001, 2024-09-17T23:29:21.690 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097250+0000", 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097250+0000", 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097251+0000", 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.043e-06 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097269+0000", 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8298e-05 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097283+0000", 2024-09-17T23:29:21.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3429e-05 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097290+0000", 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3440000000000003e-06 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097343+0000", 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3297999999999997e-05 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.097366+0000", 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2520000000000001e-05 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.099306+0000", 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001940272 2024-09-17T23:29:21.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.099356+0000", 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9456000000000002e-05 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4468.0:972 3.1e 3:78b732d7:::rbd_data.11522ab9a303.0000000000000004:head [write 15073280~28160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.452930+0000", 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.18498951399999999, 2024-09-17T23:29:21.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0041827549999999998, 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4468", 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 972 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.452930+0000", 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.452930+0000", 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.452931+0000", 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4099999999999997e-07 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.452967+0000", 2024-09-17T23:29:21.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6550000000000001e-05 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.452980+0000", 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2193e-05 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.452996+0000", 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5943e-05 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453041+0000", 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5714999999999999e-05 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453090+0000", 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8606000000000002e-05 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453251+0000", 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000160889 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454106+0000", 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00085530800000000004 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.457054+0000", 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029483690000000002 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.457082+0000", 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7741000000000001e-05 2024-09-17T23:29:21.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.457113+0000", 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0599999999999998e-05 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4468.0:971 3.11 e25/24 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:head v 25'10, mlcod=25'8)", 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.453336+0000", 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.184583736, 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003511339, 2024-09-17T23:29:21.701 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453336+0000", 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453336+0000", 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453337+0000", 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8100000000000002e-07 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453383+0000", 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6349e-05 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453402+0000", 2024-09-17T23:29:21.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9470000000000002e-05 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453408+0000", 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0000000000000002e-06 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453460+0000", 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1654999999999999e-05 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.453486+0000", 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5899999999999999e-05 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.456821+0000", 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033351019999999999 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.456847+0000", 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6081999999999999e-05 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4468.0:978 3.1e 3:78b732d7:::rbd_data.11522ab9a303.0000000000000004:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.454786+0000", 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.18313364700000001, 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002456003, 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4468", 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 978 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454786+0000", 2024-09-17T23:29:21.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454786+0000", 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454787+0000", 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4499999999999997e-07 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454791+0000", 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6949999999999999e-06 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454801+0000", 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6509999999999996e-06 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454807+0000", 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1619999999999996e-06 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454865+0000", 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7476999999999999e-05 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.454907+0000", 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2908999999999999e-05 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.457147+0000", 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022395150000000001 2024-09-17T23:29:21.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.457188+0000", 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0599999999999998e-05 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.457242+0000", 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4348999999999998e-05 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4468.0:1046 3.1e 3:78b732d7:::rbd_data.11522ab9a303.0000000000000004:head [write 29229056~34304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.472026+0000", 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.16589308799999999, 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0041333630000000001, 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4468", 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1046 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.712 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472026+0000", 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472026+0000", 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472027+0000", 2024-09-17T23:29:21.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5600000000000001e-07 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472077+0000", 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0695999999999997e-05 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472083+0000", 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.852e-06 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472092+0000", 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0899999999999994e-06 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472186+0000", 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3935000000000001e-05 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472283+0000", 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6588000000000004e-05 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472430+0000", 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014669100000000001 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473674+0000", 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001244364 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476070+0000", 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0023956960000000001 2024-09-17T23:29:21.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476103+0000", 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3534000000000001e-05 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476160+0000", 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6561000000000003e-05 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4468.0:1047 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:head [write 30833152~64000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.472104+0000", 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.165815137, 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0067606510000000003, 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4468", 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1047 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.718 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472104+0000", 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472104+0000", 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472105+0000", 2024-09-17T23:29:21.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7300000000000002e-07 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472166+0000", 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1221000000000002e-05 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472172+0000", 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6500000000000001e-06 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472175+0000", 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4740000000000001e-06 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473045+0000", 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000870041 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473133+0000", 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7719000000000005e-05 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473254+0000", 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012146000000000001 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.478684+0000", 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054299860000000004 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.478804+0000", 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012013799999999999 2024-09-17T23:29:21.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.478842+0000", 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7839999999999997e-05 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.478865+0000", 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2748999999999999e-05 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4468.0:1049 3.1e 3:78b732d7:::rbd_data.11522ab9a303.0000000000000004:head [read 30822400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.472326+0000", 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.16559342199999999, 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0040470569999999997, 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4468", 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1049 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472326+0000", 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472326+0000", 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472327+0000", 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0299999999999998e-07 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472331+0000", 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0799999999999999e-06 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472342+0000", 2024-09-17T23:29:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0624e-05 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472352+0000", 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9769999999999995e-06 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473621+0000", 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012694480000000001 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:29:21.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473665+0000", 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4172999999999999e-05 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476206+0000", 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025403029999999998 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476284+0000", 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8662e-05 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476373+0000", 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8986999999999994e-05 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4468.0:1048 3.19 e25/19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:head v 25'5, mlcod=25'3)", 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.472450+0000", 2024-09-17T23:29:21.729 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.16546905000000001, 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035898480000000001, 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472450+0000", 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472450+0000", 2024-09-17T23:29:21.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472451+0000", 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1699999999999997e-07 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472472+0000", 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0696e-05 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472481+0000", 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4790000000000002e-06 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472488+0000", 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5710000000000003e-06 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472545+0000", 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7117999999999998e-05 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472567+0000", 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1931000000000001e-05 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.475993+0000", 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0034260990000000002 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476040+0000", 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7237000000000002e-05 2024-09-17T23:29:21.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4468.0:1045 3.11 e25/24 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:head v 25'11, mlcod=25'9)", 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.472651+0000", 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.16526866000000001, 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0034302600000000001, 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472651+0000", 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472651+0000", 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472652+0000", 2024-09-17T23:29:21.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7999999999999995e-07 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472670+0000", 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8065000000000001e-05 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472682+0000", 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2483999999999999e-05 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472691+0000", 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8589999999999994e-06 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472746+0000", 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5306999999999997e-05 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472767+0000", 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1234999999999999e-05 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476043+0000", 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0032757680000000001 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476081+0000", 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7861999999999998e-05 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.738 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:783 3.b 3:d13f6fa8:::rbd_header.11522ab9a303:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.479409+0000", 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.158510225, 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021636099999999998, 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 783 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:21.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479409+0000", 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479409+0000", 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479410+0000", 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1399999999999997e-07 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479414+0000", 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2010000000000001e-06 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479422+0000", 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5549999999999997e-06 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479427+0000", 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4539999999999998e-06 2024-09-17T23:29:21.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479467+0000", 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9740000000000002e-05 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.479543+0000", 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6094999999999996e-05 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.481573+0000", 2024-09-17T23:29:21.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020299509999999999 2024-09-17T23:29:21.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.743 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:21.743 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:21.743 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:21.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:21.745 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:21.742+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:21.843 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:29:21.845 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:21.842+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:21.897 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:21.898 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:21.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:21.948 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:21.945+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:22.048 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:22.045+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:22.055 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:22.055 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:22.055 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:22.055 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:22.055 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:22.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:22.148 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:22.145+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:95 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.216 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.754301+0000", 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.4581360210000001, 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0041049040000000004, 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 95 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754301+0000", 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754301+0000", 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754302+0000", 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3799999999999996e-07 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754306+0000", 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8120000000000001e-06 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754316+0000", 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0145e-05 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754322+0000", 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6159999999999998e-06 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754385+0000", 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2848999999999994e-05 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754480+0000", 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5136999999999994e-05 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754914+0000", 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00043329499999999999 2024-09-17T23:29:22.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756733+0000", 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018191710000000001 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758301+0000", 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001567941 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758360+0000", 2024-09-17T23:29:22.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9267e-05 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758406+0000", 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5933000000000002e-05 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:96 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.754485+0000", 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.4579520540000002, 2024-09-17T23:29:22.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042234109999999998, 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 96 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754485+0000", 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754485+0000", 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754486+0000", 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5000000000000002e-07 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754491+0000", 2024-09-17T23:29:22.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8040000000000001e-06 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754500+0000", 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1249999999999999e-06 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754514+0000", 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4634000000000001e-05 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755418+0000", 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00090352100000000001 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755467+0000", 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9335000000000001e-05 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.755673+0000", 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020558999999999999 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758636+0000", 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029635030000000001 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758652+0000", 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5194999999999999e-05 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758684+0000", 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2827000000000003e-05 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758709+0000", 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4127000000000001e-05 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:97 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.754575+0000", 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.4578618699999999, 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0067742569999999997, 2024-09-17T23:29:22.228 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 97 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754575+0000", 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754575+0000", 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754576+0000", 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4499999999999996e-07 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754589+0000", 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2672e-05 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754603+0000", 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3903000000000001e-05 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754615+0000", 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1922999999999999e-05 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756053+0000", 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014387270000000001 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756140+0000", 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6750999999999997e-05 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756431+0000", 2024-09-17T23:29:22.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00029059799999999998 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758717+0000", 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022867719999999998 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761261+0000", 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00254404 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761311+0000", 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9347e-05 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761350+0000", 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8779000000000003e-05 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:235 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.921777+0000", 2024-09-17T23:29:22.234 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.29066035, 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039416529999999998, 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 235 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921777+0000", 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921777+0000", 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921778+0000", 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9100000000000002e-07 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921782+0000", 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6589999999999998e-06 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921793+0000", 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0903999999999999e-05 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921801+0000", 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8800000000000008e-06 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921854+0000", 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2549000000000001e-05 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921908+0000", 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3879999999999999e-05 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922109+0000", 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000201864 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.923163+0000", 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010535270000000001 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925643+0000", 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024804530000000001 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925691+0000", 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7206000000000002e-05 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925718+0000", 2024-09-17T23:29:22.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7840000000000001e-05 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:236 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.921822+0000", 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.2906151970000002, 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042098480000000004, 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.241 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 236 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921822+0000", 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921822+0000", 2024-09-17T23:29:22.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921822+0000", 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921826+0000", 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9999999999999998e-06 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921832+0000", 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8139999999999997e-06 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.921836+0000", 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3349999999999998e-06 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922426+0000", 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00058998799999999995 2024-09-17T23:29:22.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922513+0000", 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7163000000000006e-05 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.922845+0000", 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033201299999999998 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925745+0000", 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029001410000000002 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925966+0000", 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00022115500000000001 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.925994+0000", 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.777e-05 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.926032+0000", 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8019000000000003e-05 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:298 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.975179+0000", 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.2372584280000001, 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039826999999999996, 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 298 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975179+0000", 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975179+0000", 2024-09-17T23:29:22.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975179+0000", 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975182+0000", 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6520000000000002e-06 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975188+0000", 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6210000000000001e-06 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975191+0000", 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3749999999999999e-06 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975695+0000", 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00050367800000000002 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975740+0000", 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5179999999999998e-05 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.975968+0000", 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00022805500000000001 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.978920+0000", 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029520599999999998 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.979080+0000", 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015962600000000001 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.979120+0000", 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0516000000000003e-05 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.979161+0000", 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1390999999999997e-05 2024-09-17T23:29:22.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:299 3.7 3:e64865bb:::rbd_data.1152f108f2b3.0000000000000031:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2662400~26624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.976918+0000", 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.235518828, 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049635770000000003, 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 299 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976918+0000", 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976918+0000", 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976919+0000", 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6800000000000009e-07 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976946+0000", 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6990000000000001e-05 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976957+0000", 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0927000000000001e-05 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.976962+0000", 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999997e-06 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977003+0000", 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1099000000000001e-05 2024-09-17T23:29:22.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977079+0000", 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5844000000000001e-05 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.977157+0000", 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8085999999999997e-05 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.981645+0000", 2024-09-17T23:29:22.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044872929999999998 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.981778+0000", 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013314300000000001 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.981837+0000", 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9119000000000001e-05 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.981882+0000", 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5055000000000001e-05 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:446 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.090156+0000", 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.1222813710000001, 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039032559999999999, 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 446 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090156+0000", 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090156+0000", 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090156+0000", 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.65e-07 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090159+0000", 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4480000000000001e-06 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090165+0000", 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7580000000000003e-06 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090169+0000", 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.473e-06 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090738+0000", 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00056926000000000004 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090833+0000", 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4772000000000007e-05 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091151+0000", 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00031765299999999999 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.093872+0000", 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027213210000000001 2024-09-17T23:29:22.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.093992+0000", 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012043400000000001 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.094023+0000", 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0451e-05 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.094059+0000", 2024-09-17T23:29:22.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6321e-05 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:447 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.090254+0000", 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.1221827129999999, 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.006586087, 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 447 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090254+0000", 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090254+0000", 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090255+0000", 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090259+0000", 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090267+0000", 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8809999999999996e-06 2024-09-17T23:29:22.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.090273+0000", 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5749999999999997e-06 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091451+0000", 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011776849999999999 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091542+0000", 2024-09-17T23:29:22.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0859000000000005e-05 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.091755+0000", 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021336 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.096570+0000", 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048152309999999997 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.096742+0000", 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000172141 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.096798+0000", 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5260999999999998e-05 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.096841+0000", 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2809000000000003e-05 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.270 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:473 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112657+0000", 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0997804420000001, 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0060128910000000002, 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 473 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112657+0000", 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112657+0000", 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112658+0000", 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6700000000000002e-07 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112661+0000", 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8739999999999997e-06 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112671+0000", 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8970000000000003e-06 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112679+0000", 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9889999999999992e-06 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.114080+0000", 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014005199999999999 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.114134+0000", 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4586999999999998e-05 2024-09-17T23:29:22.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.114392+0000", 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00025784799999999997 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117000+0000", 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026077560000000001 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118567+0000", 2024-09-17T23:29:22.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001567225 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118630+0000", 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2664999999999997e-05 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118670+0000", 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9663000000000003e-05 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.276 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:474 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112692+0000", 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0997454539999998, 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0061299800000000002, 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 474 2024-09-17T23:29:22.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112692+0000", 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112692+0000", 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112692+0000", 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112697+0000", 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5349999999999998e-06 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112704+0000", 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.126e-06 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112710+0000", 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5509999999999999e-06 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.114992+0000", 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022827540000000001 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.115083+0000", 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1147e-05 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.115476+0000", 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0003927 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118719+0000", 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0032425129999999998 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118768+0000", 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8968000000000003e-05 2024-09-17T23:29:22.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118793+0000", 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5318000000000001e-05 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118822+0000", 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8799e-05 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:475 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112725+0000", 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0997124010000001, 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0061485639999999996, 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 475 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112725+0000", 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112725+0000", 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112725+0000", 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6499999999999999e-07 2024-09-17T23:29:22.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112729+0000", 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5020000000000001e-06 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112737+0000", 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1620000000000008e-06 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112742+0000", 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3890000000000004e-06 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.116003+0000", 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0032602130000000001 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.116088+0000", 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5696000000000001e-05 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.116588+0000", 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00049983899999999997 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118759+0000", 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021703540000000002 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118829+0000", 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0349e-05 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118853+0000", 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3757999999999999e-05 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118873+0000", 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0737000000000002e-05 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:476 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.288 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112750+0000", 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0996867950000002, 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062919120000000002, 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 476 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112750+0000", 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112750+0000", 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112751+0000", 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1600000000000002e-07 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112754+0000", 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9450000000000002e-06 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112772+0000", 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8578000000000001e-05 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112778+0000", 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117041+0000", 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042622830000000004 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117122+0000", 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1048000000000001e-05 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117510+0000", 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038810100000000002 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118880+0000", 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013704730000000001 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118939+0000", 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8912000000000001e-05 2024-09-17T23:29:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118998+0000", 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9082000000000001e-05 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119042+0000", 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4140000000000001e-05 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:477 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0996447919999999, 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081640670000000005, 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 477 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.295 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112793+0000", 2024-09-17T23:29:22.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2e-07 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112796+0000", 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9620000000000002e-06 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112803+0000", 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5989999999999996e-06 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112809+0000", 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.276e-06 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117872+0000", 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050631349999999999 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117960+0000", 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8399000000000005e-05 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118168+0000", 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020731300000000001 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120846+0000", 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026780630000000001 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120889+0000", 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2904000000000001e-05 2024-09-17T23:29:22.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120920+0000", 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1075999999999999e-05 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120956+0000", 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6820000000000003e-05 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:478 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0996101199999999, 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0088678699999999999, 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 478 2024-09-17T23:29:22.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2599999999999998e-07 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112831+0000", 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1669999999999999e-06 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112839+0000", 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4230000000000005e-06 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112845+0000", 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.02e-06 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119096+0000", 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062507580000000004 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119178+0000", 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2100000000000003e-05 2024-09-17T23:29:22.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119698+0000", 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00051989400000000002 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121193+0000", 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014956279999999999 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121598+0000", 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00040432800000000002 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121653+0000", 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5541999999999998e-05 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121695+0000", 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1584e-05 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.306 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:479 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0995185520000001, 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010595982, 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 479 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3900000000000004e-07 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112923+0000", 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5379999999999998e-06 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112931+0000", 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5390000000000002e-06 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112936+0000", 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4809999999999997e-06 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120113+0000", 2024-09-17T23:29:22.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0071769390000000002 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120187+0000", 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3967000000000005e-05 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120554+0000", 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036718900000000002 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121301+0000", 2024-09-17T23:29:22.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00074710800000000002 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123418+0000", 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021164729999999998 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123475+0000", 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7598e-05 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123515+0000", 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9310999999999997e-05 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:480 3.6 3:641496d6:::rbd_data.1152f108f2b3.0000000000000036:head [read 1069056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.099156319, 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079137459999999993, 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 480 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113282+0000", 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3e-07 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113286+0000", 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.228e-06 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113295+0000", 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.38e-06 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113300+0000", 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0010000000000001e-06 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120998+0000", 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0076977790000000001 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121079+0000", 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1055000000000002e-05 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121195+0000", 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115573 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:487 3.8 3:12e1d38a:::rbd_data.1152f108f2b3.0000000000000015:head [set-alloc-hint object_size 4194304 write_size 4194304,write 864256~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.317 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.118996+0000", 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0934408449999999, 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0043977290000000004, 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 487 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118996+0000", 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118996+0000", 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118997+0000", 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2200000000000002e-07 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119065+0000", 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7398999999999996e-05 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119095+0000", 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9887999999999999e-05 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119101+0000", 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8789999999999997e-06 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119164+0000", 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2506000000000001e-05 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119403+0000", 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00023890399999999999 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119675+0000", 2024-09-17T23:29:22.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00027206000000000002 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120753+0000", 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001078138 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123293+0000", 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025402509999999999 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123351+0000", 2024-09-17T23:29:22.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7503000000000002e-05 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123394+0000", 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3278999999999997e-05 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:499 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.121886+0000", 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.0905510390000002, 2024-09-17T23:29:22.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003984296, 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 499 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121886+0000", 2024-09-17T23:29:22.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121886+0000", 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121887+0000", 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8899999999999998e-07 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121892+0000", 2024-09-17T23:29:22.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1390000000000003e-06 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121904+0000", 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1933000000000001e-05 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121919+0000", 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5031000000000001e-05 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.122459+0000", 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00053992899999999997 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.122547+0000", 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8213000000000004e-05 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.122835+0000", 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028779800000000003 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.125773+0000", 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029379419999999998 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.125789+0000", 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6218e-05 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.125835+0000", 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5926e-05 2024-09-17T23:29:22.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.125870+0000", 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5377999999999999e-05 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4468.0:1048 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:head [write 10849792~11264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:21.472135+0000", 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.74030212699999998, 2024-09-17T23:29:22.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0041419430000000004, 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4468", 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1048 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472135+0000", 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472135+0000", 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472136+0000", 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.04e-06 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472150+0000", 2024-09-17T23:29:22.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3789e-05 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472163+0000", 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-05 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472171+0000", 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6790000000000005e-06 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472232+0000", 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.122e-05 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472280+0000", 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7275999999999999e-05 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.472385+0000", 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000105449 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.473232+0000", 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00084682500000000005 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476171+0000", 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029388550000000002 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476231+0000", 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0105000000000001e-05 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:21.476277+0000", 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6125000000000003e-05 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.335 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:22.336 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T23:29:22.337 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:22.337 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:22.335+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:22.440 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:22.436+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:22.494 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:22.494 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:22.495 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:22.495 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:22.503 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:22.538 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:22.536+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:22.619 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:22.619 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:22.619 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:22.619 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:22.619 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:22.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:22.672 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:22.669+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:69 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'43, mlcod=13'39)", 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.753333+0000", 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.981709872, 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050282740000000001, 2024-09-17T23:29:22.738 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753333+0000", 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753333+0000", 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753334+0000", 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5300000000000004e-07 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753403+0000", 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9919000000000006e-05 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753414+0000", 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0271e-05 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753419+0000", 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8199999999999996e-06 2024-09-17T23:29:22.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753467+0000", 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8306000000000001e-05 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.753480+0000", 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3233e-05 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.758323+0000", 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048426099999999998 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.758361+0000", 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8562000000000001e-05 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.744 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:73 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'47, mlcod=13'43)", 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.760587+0000", 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.974455894999998, 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055840430000000003, 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760587+0000", 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760587+0000", 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760588+0000", 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4200000000000005e-07 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760633+0000", 2024-09-17T23:29:22.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5021000000000001e-05 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760645+0000", 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2095e-05 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760650+0000", 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4149999999999998e-06 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760702+0000", 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1303e-05 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.760717+0000", 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4935e-05 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766133+0000", 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054167119999999997 2024-09-17T23:29:22.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766171+0000", 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7719999999999998e-05 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:74 1.0 e13/12 1:30306672:devicehealth::main.db.0000000000000000:head v 13'48, mlcod=13'43)", 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.761082+0000", 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.973960863999999, 2024-09-17T23:29:22.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051069449999999999, 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761082+0000", 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761082+0000", 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761083+0000", 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199999999999999e-07 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761088+0000", 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.062e-06 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761099+0000", 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1569999999999999e-05 2024-09-17T23:29:22.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.761106+0000", 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.63e-06 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.762999+0000", 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018933380000000001 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.763018+0000", 2024-09-17T23:29:22.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8431e-05 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766173+0000", 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031556700000000002 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.766189+0000", 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5461999999999999e-05 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:84 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'54, mlcod=13'50)", 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.777856+0000", 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.957186548999999, 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0052776580000000002, 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777856+0000", 2024-09-17T23:29:22.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777856+0000", 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777857+0000", 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999999999999999e-07 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777914+0000", 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7414999999999998e-05 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777924+0000", 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0067e-05 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777929+0000", 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7419999999999997e-06 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777961+0000", 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1917e-05 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.777974+0000", 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2799000000000001e-05 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783091+0000", 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051176040000000004 2024-09-17T23:29:22.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783134+0000", 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2493999999999997e-05 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:85 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'55, mlcod=13'50)", 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.956548312999999, 2024-09-17T23:29:22.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046766079999999996, 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778495+0000", 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6199999999999997e-07 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778501+0000", 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.276e-06 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778512+0000", 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1399e-05 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778518+0000", 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6840000000000001e-06 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778559+0000", 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1214000000000002e-05 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.778572+0000", 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2653999999999999e-05 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783137+0000", 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0045653730000000002 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.783171+0000", 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4245999999999997e-05 2024-09-17T23:29:22.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:112 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'74, mlcod=13'68)", 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.952526+0000", 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.782516344000001, 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0052955399999999996, 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952526+0000", 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952526+0000", 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952527+0000", 2024-09-17T23:29:22.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6599999999999996e-07 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952588+0000", 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0785999999999997e-05 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952599+0000", 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0563e-05 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952604+0000", 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952637+0000", 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2446000000000001e-05 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.952649+0000", 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425999999999999e-05 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957780+0000", 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.005131025 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957822+0000", 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1987999999999998e-05 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:113 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'75, mlcod=13'68)", 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.781987264000001, 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047959250000000004, 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953056+0000", 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6100000000000001e-07 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953060+0000", 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8530000000000002e-06 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953068+0000", 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2109999999999998e-06 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953072+0000", 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1579999999999998e-06 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953112+0000", 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9628000000000001e-05 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.953126+0000", 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4515000000000001e-05 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957825+0000", 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046981899999999997 2024-09-17T23:29:22.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:28:59.957851+0000", 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6908999999999999e-05 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:130 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'88, mlcod=13'85)", 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.431764918999999, 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055141970000000002, 2024-09-17T23:29:22.772 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303278+0000", 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.82e-07 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303283+0000", 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303291+0000", 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1550000000000007e-06 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303295+0000", 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1150000000000004e-06 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303326+0000", 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0762999999999997e-05 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303336+0000", 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122000000000001e-05 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308768+0000", 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054314560000000003 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308792+0000", 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.461e-05 2024-09-17T23:29:22.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:131 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'89, mlcod=13'85)", 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.303537+0000", 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.431505810000001, 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0083550710000000004, 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303537+0000", 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303537+0000", 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303538+0000", 2024-09-17T23:29:22.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303614+0000", 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6341000000000007e-05 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303621+0000", 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4780000000000002e-06 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303625+0000", 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5310000000000002e-06 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303654+0000", 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9006999999999999e-05 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.303664+0000", 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9150000000000008e-06 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311862+0000", 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081983720000000006 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311892+0000", 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9858000000000001e-05 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.781 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:132 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'90, mlcod=13'85)", 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.430998102, 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078664449999999997, 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304045+0000", 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8800000000000003e-07 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304049+0000", 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7290000000000002e-06 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304056+0000", 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2609999999999996e-06 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304060+0000", 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.49e-06 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304091+0000", 2024-09-17T23:29:22.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1075000000000003e-05 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304102+0000", 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1523e-05 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311894+0000", 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077915980000000003 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311911+0000", 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7280999999999999e-05 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:133 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'91, mlcod=13'85)", 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.430574103000001, 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074569090000000003, 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304469+0000", 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2e-07 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304473+0000", 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0559999999999998e-06 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304481+0000", 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5429999999999996e-06 2024-09-17T23:29:22.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304484+0000", 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304512+0000", 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7787999999999999e-05 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304524+0000", 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.129e-05 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311912+0000", 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0073882729999999999 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311926+0000", 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3856000000000001e-05 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.790 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:134 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'92, mlcod=13'85)", 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.430223013999999, 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007118615, 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304820+0000", 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304871+0000", 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0738000000000001e-05 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304878+0000", 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1409999999999999e-06 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304883+0000", 2024-09-17T23:29:22.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9069999999999998e-06 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304913+0000", 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9481000000000001e-05 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.304925+0000", 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311926+0000", 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0070011939999999996 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311938+0000", 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2279000000000001e-05 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:135 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'93, mlcod=13'85)", 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:22.795 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.429756786999999, 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0066672509999999999, 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:22.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305286+0000", 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3799999999999998e-07 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305290+0000", 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3500000000000001e-06 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305296+0000", 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305301+0000", 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0440000000000003e-06 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305329+0000", 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7994e-05 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305341+0000", 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1503e-05 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311940+0000", 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0065988460000000002 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311953+0000", 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3658999999999999e-05 2024-09-17T23:29:22.799 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:136 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'94, mlcod=13'86)", 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.305685+0000", 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.429357536000001, 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062818350000000004, 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305685+0000", 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305685+0000", 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305686+0000", 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:29:22.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305689+0000", 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.738e-06 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305696+0000", 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7089999999999996e-06 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305699+0000", 2024-09-17T23:29:22.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.252e-06 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305730+0000", 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0941000000000001e-05 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.305742+0000", 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.128e-05 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311955+0000", 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.006213105 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311967+0000", 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2368000000000001e-05 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:137 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'95, mlcod=13'86)", 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:22.804 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.428936072999999, 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058731549999999997, 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306107+0000", 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1200000000000003e-07 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306157+0000", 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9883999999999999e-05 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306165+0000", 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6459999999999993e-06 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306168+0000", 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4800000000000001e-06 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306203+0000", 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4727999999999997e-05 2024-09-17T23:29:22.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306216+0000", 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3382e-05 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311968+0000", 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057512730000000003 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311980+0000", 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.225e-05 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:138 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'96, mlcod=13'86)", 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.428452313000001, 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054006449999999999, 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.809 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306591+0000", 2024-09-17T23:29:22.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3900000000000005e-07 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306595+0000", 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0679999999999998e-06 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306603+0000", 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.678e-06 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306606+0000", 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.659e-06 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306639+0000", 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2270000000000001e-05 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306649+0000", 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0754000000000001e-05 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311980+0000", 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053309890000000004 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311991+0000", 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0688e-05 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.813 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:139 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'97, mlcod=13'86)", 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.306913+0000", 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.428129761000001, 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050921689999999997, 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306913+0000", 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306913+0000", 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306914+0000", 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0200000000000002e-07 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306920+0000", 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0569999999999998e-06 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306927+0000", 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8959999999999997e-06 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306930+0000", 2024-09-17T23:29:22.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2600000000000001e-06 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306960+0000", 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.012e-05 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.306971+0000", 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1489e-05 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.311992+0000", 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050203000000000001 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312005+0000", 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3545e-05 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:140 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'98, mlcod=13'86)", 2024-09-17T23:29:22.818 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.307277+0000", 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.427765554, 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049717499999999996, 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307277+0000", 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307277+0000", 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307278+0000", 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8699999999999995e-07 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307317+0000", 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9165000000000002e-05 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307325+0000", 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0020000000000006e-06 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307329+0000", 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5860000000000001e-06 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307360+0000", 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1136000000000001e-05 2024-09-17T23:29:22.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307370+0000", 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0553e-05 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312220+0000", 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048497770000000004 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312249+0000", 2024-09-17T23:29:22.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9043999999999998e-05 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4192.0:141 1.0 e13/12 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 13'99, mlcod=13'86)", 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.427328195000001, 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0045645369999999996, 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307715+0000", 2024-09-17T23:29:22.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8299999999999997e-07 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307720+0000", 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3750000000000002e-06 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307728+0000", 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5259999999999996e-06 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.307731+0000", 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.433e-06 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308823+0000", 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010911499999999999 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.308835+0000", 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2627e-05 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312251+0000", 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0034158320000000002 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:00.312279+0000", 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8110999999999999e-05 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.827 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:485 3.12 3:4a45a2fa:::rbd_data.1152f108f2b3.0000000000000010:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4046848~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.118894+0000", 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.6161487709999998, 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046932019999999996, 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 485 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.828 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118894+0000", 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118894+0000", 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118896+0000", 2024-09-17T23:29:22.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.807e-06 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118967+0000", 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0833000000000004e-05 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118993+0000", 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6537999999999999e-05 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119001+0000", 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2960000000000001e-06 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119073+0000", 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2640999999999998e-05 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119198+0000", 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012501999999999999 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119329+0000", 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013078100000000001 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120554+0000", 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012250620000000001 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123491+0000", 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002936943 2024-09-17T23:29:22.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123541+0000", 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0299000000000001e-05 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123587+0000", 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5982000000000001e-05 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:22.833 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:22.834 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:22.835 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:22.833+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:22.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:22.845 INFO:teuthology.orchestra.run.smithi139.stdout:349 punch from 0x3466400 to 0x3471000, (0xac00 bytes) 2024-09-17T23:29:22.864 INFO:teuthology.orchestra.run.smithi139.stdout:350 write 0xb711400 thru 0xb71bf63 (0xab64 bytes) 2024-09-17T23:29:22.864 INFO:teuthology.orchestra.run.smithi139.stdout:352 punch from 0x27bac00 to 0x27be800, (0x3c00 bytes) 2024-09-17T23:29:22.893 INFO:teuthology.orchestra.run.smithi139.stdout:353 write 0x9107e00 thru 0x910ae54 (0x3055 bytes) 2024-09-17T23:29:22.894 INFO:teuthology.orchestra.run.smithi139.stdout:356 write 0x78e5c00 thru 0x78f3379 (0xd77a bytes) 2024-09-17T23:29:22.894 INFO:teuthology.orchestra.run.smithi139.stdout:357 punch from 0x72b1800 to 0x72be400, (0xcc00 bytes) 2024-09-17T23:29:22.933 INFO:teuthology.orchestra.run.smithi139.stdout:358 trunc from 0xebf7400 to 0x9bace00 2024-09-17T23:29:22.936 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:22.933+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:22.954 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:22.954 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:22.954 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:22.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:22.998 INFO:teuthology.orchestra.run.smithi139.stdout:359 read 0x5b31200 thru 0x5b31314 (0x115 bytes) 2024-09-17T23:29:22.999 INFO:teuthology.orchestra.run.smithi139.stdout:360 write 0x39bc00 thru 0x3a7f14 (0xc315 bytes) 2024-09-17T23:29:23.000 INFO:teuthology.orchestra.run.smithi139.stdout:361 punch from 0x47b6800 to 0x47b8000, (0x1800 bytes) 2024-09-17T23:29:23.002 INFO:teuthology.orchestra.run.smithi139.stdout:362 read 0x3f9c800 thru 0x3faabb0 (0xe3b1 bytes) 2024-09-17T23:29:23.004 INFO:teuthology.orchestra.run.smithi139.stdout:363 read 0x3aeb400 thru 0x3af3be0 (0x87e1 bytes) 2024-09-17T23:29:23.004 INFO:teuthology.orchestra.run.smithi139.stdout:367 punch from 0x8891600 to 0x8898800, (0x7200 bytes) 2024-09-17T23:29:23.004 INFO:teuthology.orchestra.run.smithi139.stdout:369 read 0x1452c00 thru 0x145ce44 (0xa245 bytes) 2024-09-17T23:29:23.004 INFO:teuthology.orchestra.run.smithi139.stdout:370 write 0x2d5d400 thru 0x2d607bf (0x33c0 bytes) 2024-09-17T23:29:23.005 INFO:teuthology.orchestra.run.smithi139.stdout:372 write 0x67dcc00 thru 0x67e7c34 (0xb035 bytes) 2024-09-17T23:29:23.005 INFO:teuthology.orchestra.run.smithi139.stdout:377 read 0x1269000 thru 0x126d798 (0x4799 bytes) 2024-09-17T23:29:23.006 INFO:teuthology.orchestra.run.smithi139.stdout:379 write 0x6f30c00 thru 0x6f38ff7 (0x83f8 bytes) 2024-09-17T23:29:23.006 INFO:teuthology.orchestra.run.smithi139.stdout:380 read 0x6a17e00 thru 0x6a242b0 (0xc4b1 bytes) 2024-09-17T23:29:23.007 INFO:teuthology.orchestra.run.smithi139.stdout:382 read 0x4de8800 thru 0x4defe34 (0x7635 bytes) 2024-09-17T23:29:23.007 INFO:teuthology.orchestra.run.smithi139.stdout:384 read 0x31bdc00 thru 0x31c0b60 (0x2f61 bytes) 2024-09-17T23:29:23.008 INFO:teuthology.orchestra.run.smithi139.stdout:387 read 0x91da400 thru 0x91db206 (0xe07 bytes) 2024-09-17T23:29:23.008 INFO:teuthology.orchestra.run.smithi139.stdout:388 read 0x5f35e00 thru 0x5f3e800 (0x8a01 bytes) 2024-09-17T23:29:23.011 INFO:teuthology.orchestra.run.smithi139.stdout:389 trunc from 0x9bace00 to 0xc8ebc00 2024-09-17T23:29:23.037 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:23.035+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:23.061 INFO:teuthology.orchestra.run.smithi139.stdout:390 write 0xd865400 thru 0xd8663fb (0xffc bytes) 2024-09-17T23:29:23.061 INFO:teuthology.orchestra.run.smithi139.stdout:391 read 0x3532a00 thru 0x35351e6 (0x27e7 bytes) 2024-09-17T23:29:23.062 INFO:teuthology.orchestra.run.smithi139.stdout:393 read 0x29aa600 thru 0x29ae7b4 (0x41b5 bytes) 2024-09-17T23:29:23.072 INFO:teuthology.orchestra.run.smithi139.stdout:394 write 0xe1b0800 thru 0xe1b97fb (0x8ffc bytes) 2024-09-17T23:29:23.073 INFO:teuthology.orchestra.run.smithi139.stdout:397 read 0x6225e00 thru 0x6226653 (0x854 bytes) 2024-09-17T23:29:23.073 INFO:teuthology.orchestra.run.smithi139.stdout:398 read 0x8f2d400 thru 0x8f38ce0 (0xb8e1 bytes) 2024-09-17T23:29:23.083 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:23.083 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:23.089 INFO:teuthology.orchestra.run.smithi139.stdout:402 trunc from 0xe1b9800 to 0xb59e00 2024-09-17T23:29:23.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:23.145 INFO:teuthology.orchestra.run.smithi139.stdout:404 punch from 0x94fc00 to 0x955600, (0x5a00 bytes) 2024-09-17T23:29:23.161 INFO:teuthology.orchestra.run.smithi139.stdout:405 write 0x4662200 thru 0x4664c6a (0x2a6b bytes) 2024-09-17T23:29:23.162 INFO:teuthology.orchestra.run.smithi139.stdout:406 write 0x409d600 thru 0x40a9434 (0xbe35 bytes) 2024-09-17T23:29:23.177 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:23.176+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:23.192 INFO:teuthology.orchestra.run.smithi139.stdout:408 write 0xeb48000 thru 0xeb51320 (0x9321 bytes) 2024-09-17T23:29:23.193 INFO:teuthology.orchestra.run.smithi139.stdout:409 write 0x7631c00 thru 0x76369a7 (0x4da8 bytes) 2024-09-17T23:29:23.193 INFO:teuthology.orchestra.run.smithi139.stdout:410 read 0x6636400 thru 0x66415e4 (0xb1e5 bytes) 2024-09-17T23:29:23.194 INFO:teuthology.orchestra.run.smithi139.stdout:412 punch from 0x819400 to 0x824a00, (0xb600 bytes) 2024-09-17T23:29:23.194 INFO:teuthology.orchestra.run.smithi139.stdout:414 read 0x6875a00 thru 0x6877464 (0x1a65 bytes) 2024-09-17T23:29:23.198 INFO:teuthology.orchestra.run.smithi139.stdout:416 read 0x744a000 thru 0x744a286 (0x287 bytes) 2024-09-17T23:29:23.198 INFO:teuthology.orchestra.run.smithi139.stdout:417 write 0x61a400 thru 0x627028 (0xcc29 bytes) 2024-09-17T23:29:23.199 INFO:teuthology.orchestra.run.smithi139.stdout:422 write 0x32bda00 thru 0x32c94f1 (0xbaf2 bytes) 2024-09-17T23:29:23.200 INFO:teuthology.orchestra.run.smithi139.stdout:423 read 0x9903000 thru 0x9909f75 (0x6f76 bytes) 2024-09-17T23:29:23.211 INFO:teuthology.orchestra.run.smithi139.stdout:427 trunc from 0xeb51400 to 0x4f12e00 2024-09-17T23:29:23.239 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:23.237+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:23.240 INFO:teuthology.orchestra.run.smithi139.stdout:436 punch from 0x4c07400 to 0x4c12200, (0xae00 bytes) 2024-09-17T23:29:23.243 INFO:teuthology.orchestra.run.smithi139.stdout:437 write 0x2398c00 thru 0x239b89d (0x2c9e bytes) 2024-09-17T23:29:23.243 INFO:teuthology.orchestra.run.smithi139.stdout:439 read 0x22e9c00 thru 0x22f864b (0xea4c bytes) 2024-09-17T23:29:23.246 INFO:teuthology.orchestra.run.smithi139.stdout:444 trunc from 0x4f12e00 to 0x354c400 2024-09-17T23:29:23.252 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:23.252 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:06.500715+0000", 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.748140606, 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018575569999999999, 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:23.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5 2024-09-17T23:29:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500715+0000", 2024-09-17T23:29:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500715+0000", 2024-09-17T23:29:23.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500717+0000", 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.035e-06 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500735+0000", 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8303000000000001e-05 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500753+0000", 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7532000000000001e-05 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500763+0000", 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8330000000000006e-06 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500817+0000", 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4784999999999998e-05 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.500940+0000", 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000122803 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.501065+0000", 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012492100000000001 2024-09-17T23:29:23.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.501849+0000", 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078371500000000004 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502450+0000", 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060106700000000005 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502514+0000", 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4523000000000005e-05 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:06.502572+0000", 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.804e-05 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T23:29:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.746623499, 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046920809999999999, 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502234+0000", 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9400000000000001e-06 2024-09-17T23:29:23.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502243+0000", 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4290000000000007e-06 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502260+0000", 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6872e-05 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502272+0000", 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1792e-05 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502599+0000", 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032732099999999999 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502674+0000", 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.436e-05 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502795+0000", 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121549 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.504847+0000", 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020513620000000002 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506847+0000", 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020007689999999999 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506887+0000", 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9647000000000002e-05 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506924+0000", 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7039999999999998e-05 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:175 3.b e23/19 3:d3468d19:::rbd_data.1152f108f2b3.0000000000000009:head v 23'2, mlcod=0'0)", 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3915064059999995, 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031410980000000002, 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857350+0000", 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857354+0000", 2024-09-17T23:29:23.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1520000000000002e-06 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857361+0000", 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8299999999999996e-06 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857367+0000", 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857408+0000", 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1065000000000001e-05 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857425+0000", 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6977999999999999e-05 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.860455+0000", 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030295080000000002 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.860490+0000", 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5401999999999997e-05 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:300 3.9 3:959da7de:::rbd_data.1152f108f2b3.0000000000000017:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2336768~15360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.980034+0000", 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2688217430000002, 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021285380000000001, 2024-09-17T23:29:23.272 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 300 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980034+0000", 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980034+0000", 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980035+0000", 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5829999999999999e-06 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980084+0000", 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8656999999999998e-05 2024-09-17T23:29:23.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980098+0000", 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3750999999999999e-05 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980107+0000", 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7059999999999999e-06 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980162+0000", 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5052000000000001e-05 2024-09-17T23:29:23.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980255+0000", 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2057000000000003e-05 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.980344+0000", 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9415999999999999e-05 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.981077+0000", 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073300600000000002 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.982049+0000", 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00097169599999999997 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.982107+0000", 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8322999999999997e-05 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.982162+0000", 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5291000000000003e-05 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:355 3.9 3:91b1f5bf:::rbd_data.1152f108f2b3.0000000000000029:head [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2353741859999996, 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029455660000000002, 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 355 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013482+0000", 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0529999999999999e-06 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013494+0000", 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1239e-05 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013508+0000", 2024-09-17T23:29:23.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4321000000000001e-05 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013518+0000", 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8320000000000001e-06 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013564+0000", 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6347999999999997e-05 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013617+0000", 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2679000000000002e-05 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.016427+0000", 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002810094 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:385 3.9 3:91b1f5bf:::rbd_data.1152f108f2b3.0000000000000029:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:23.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.044987+0000", 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.2038689050000002, 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019881590000000002, 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 385 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044987+0000", 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044987+0000", 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044987+0000", 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2799999999999996e-07 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044991+0000", 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1620000000000001e-06 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045001+0000", 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5370000000000003e-06 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045007+0000", 2024-09-17T23:29:23.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3840000000000002e-06 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045496+0000", 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048869799999999998 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045575+0000", 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9385e-05 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045685+0000", 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109558 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046482+0000", 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079742400000000003 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046919+0000", 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00043692699999999999 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046949+0000", 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0127000000000001e-05 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046975+0000", 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5329000000000001e-05 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4468.0:605 3.b e25/19 3:d1f3e708:::rbd_data.11522ab9a303.0000000000000005:head v 25'21, mlcod=25'19)", 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:21.309518+0000", 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.9393372800000002, 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024615079999999998, 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309518+0000", 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309518+0000", 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309519+0000", 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309543+0000", 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.427e-05 2024-09-17T23:29:23.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309557+0000", 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3342e-05 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309573+0000", 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6280999999999999e-05 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309629+0000", 2024-09-17T23:29:23.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5869000000000003e-05 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.309651+0000", 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2269999999999999e-05 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.311940+0000", 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022892609999999999 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.311980+0000", 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9443999999999997e-05 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4468.0:910 3.f 3:f7e9d27a:::rbd_object_map.11522ab9a303:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.821269456, 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003778319, 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4468", 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 910 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427587+0000", 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999998e-07 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427591+0000", 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9160000000000003e-06 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427600+0000", 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5740000000000002e-06 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427606+0000", 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9220000000000002e-06 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427654+0000", 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8059999999999997e-05 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427748+0000", 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3789999999999995e-05 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.430511+0000", 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027624400000000001 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431143+0000", 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063282300000000002 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431309+0000", 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016585999999999999 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431340+0000", 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0731000000000001e-05 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431364+0000", 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.44e-05 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:750 3.15 3:af4fb69a:::rbd_data.1152fda325c3.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 16145408~55296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:22.908210+0000", 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.34064588800000001, 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020183699999999998, 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 750 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908210+0000", 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908210+0000", 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908211+0000", 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2669999999999999e-06 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908317+0000", 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106364 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908329+0000", 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1416999999999999e-05 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908334+0000", 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0980000000000001e-06 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908393+0000", 2024-09-17T23:29:23.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908541+0000", 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148097 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.908769+0000", 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022823299999999999 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.909818+0000", 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00104941 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.910129+0000", 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000310193 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.910183+0000", 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:22.910228+0000", 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4956999999999997e-05 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2049 3.a 3:52cea45b:::rbd_data.1152fda325c3.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15394839499999999, 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002789297, 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2049 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094908+0000", 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094911+0000", 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-06 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094919+0000", 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8009999999999993e-06 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094924+0000", 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4469999999999999e-06 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094967+0000", 2024-09-17T23:29:23.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3343000000000002e-05 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.095505+0000", 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053770000000000001 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.095622+0000", 2024-09-17T23:29:23.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117087 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.096354+0000", 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073242800000000003 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097606+0000", 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001251432 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097660+0000", 2024-09-17T23:29:23.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3693000000000002e-05 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097696+0000", 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6807000000000002e-05 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2333 3.17 e26/19 3:eee396ce:::rbd_object_map.1152fda325c3:head v 26'37, mlcod=26'35)", 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.137126+0000", 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.11172989799999999, 2024-09-17T23:29:23.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019877610000000002, 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137126+0000", 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137126+0000", 2024-09-17T23:29:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137126+0000", 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137131+0000", 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4469999999999999e-06 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137140+0000", 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3740000000000004e-06 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137145+0000", 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5179999999999998e-06 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137196+0000", 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1221000000000003e-05 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.137209+0000", 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3028e-05 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.139083+0000", 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001873783 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.139113+0000", 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0766999999999999e-05 2024-09-17T23:29:23.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2378 3.c 3:36bb075d:::rbd_header.1152fda325c3:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.095119455000000006, 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024627289999999999, 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2378 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153737+0000", 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5600000000000005e-07 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153740+0000", 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0319999999999999e-06 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153746+0000", 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-17T23:29:23.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153750+0000", 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6710000000000001e-06 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153784+0000", 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4799999999999999e-05 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153830+0000", 2024-09-17T23:29:23.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5719000000000001e-05 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153947+0000", 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011649100000000001 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.154698+0000", 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075154900000000001 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156144+0000", 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445794 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156173+0000", 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9436000000000001e-05 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156199+0000", 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-05 2024-09-17T23:29:23.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2455 3.c 3:36bb075d:::rbd_header.1152fda325c3:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.183615+0000", 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.065240498999999993, 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024458169999999999, 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:23.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2455 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183615+0000", 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183615+0000", 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183616+0000", 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5600000000000005e-07 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183619+0000", 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6200000000000001e-06 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183628+0000", 2024-09-17T23:29:23.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1560000000000001e-06 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183633+0000", 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5480000000000003e-06 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183682+0000", 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8934999999999998e-05 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183765+0000", 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3014000000000002e-05 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.183952+0000", 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000187536 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.184717+0000", 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076473100000000005 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.185958+0000", 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012409630000000001 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.186016+0000", 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8041999999999998e-05 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.186061+0000", 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4616000000000001e-05 2024-09-17T23:29:23.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2477 3.17 e26/19 3:eee396ce:::rbd_object_map.1152fda325c3:head v 26'40, mlcod=26'38)", 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055349430999999998, 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034184390000000001, 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-17T23:29:23.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193514+0000", 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193521+0000", 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1230000000000002e-06 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193526+0000", 2024-09-17T23:29:23.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1780000000000002e-06 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193596+0000", 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9776000000000004e-05 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193618+0000", 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2603e-05 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196879+0000", 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032604230000000001 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196924+0000", 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5816000000000003e-05 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2478 3.12 e26/19 3:4952c6c2:::rbd_data.1152fda325c3.0000000000000000:head v 26'9, mlcod=26'7)", 2024-09-17T23:29:23.332 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.055233474999999997, 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032918359999999998, 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193652+0000", 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9592000000000001e-05 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193658+0000", 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9669999999999998e-06 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193662+0000", 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6959999999999998e-06 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193711+0000", 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9685000000000003e-05 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193724+0000", 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2999999999999999e-05 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196886+0000", 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031617390000000002 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196914+0000", 2024-09-17T23:29:23.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7724000000000001e-05 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2529 3.15 3:af4fb69a:::rbd_data.1152fda325c3.0000000000000001:head [stat,truncate 10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025695198999999998, 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054525839999999999, 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2529 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:23.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223161+0000", 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7299999999999996e-07 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223164+0000", 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0709999999999999e-06 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223172+0000", 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0450000000000001e-06 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223176+0000", 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2950000000000004e-06 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223233+0000", 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6994999999999999e-05 2024-09-17T23:29:23.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223318+0000", 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4400999999999994e-05 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223467+0000", 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149796 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224157+0000", 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000689465 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228516+0000", 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004358682 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228573+0000", 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7596999999999998e-05 2024-09-17T23:29:23.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228613+0000", 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9663999999999998e-05 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2528 3.12 e26/19 3:4952c6c2:::rbd_data.1152fda325c3.0000000000000000:head v 26'10, mlcod=26'8)", 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223374+0000", 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025481612000000001, 2024-09-17T23:29:23.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023484280000000001, 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223374+0000", 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223374+0000", 2024-09-17T23:29:23.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223374+0000", 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223381+0000", 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.72e-06 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223388+0000", 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4980000000000003e-06 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223392+0000", 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0099999999999997e-06 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223427+0000", 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5707999999999998e-05 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223446+0000", 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9103e-05 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.225675+0000", 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002228608 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.225722+0000", 2024-09-17T23:29:23.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7296999999999998e-05 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2531 3.17 e26/19 3:ec02a8b5:::rbd_data.1152fda325c3.0000000000000003:head v 26'43, mlcod=26'41)", 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025410413999999999, 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051349280000000004, 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-09-17T23:29:23.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223451+0000", 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223458+0000", 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4150000000000004e-06 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223461+0000", 2024-09-17T23:29:23.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3390000000000001e-06 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223501+0000", 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0201999999999999e-05 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223521+0000", 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0085000000000002e-05 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228525+0000", 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050040179999999998 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228580+0000", 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4688999999999998e-05 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2532 3.1b e26/19 3:d90b4fac:::rbd_data.1152fda325c3.0000000000000004:head v 26'13, mlcod=26'11)", 2024-09-17T23:29:23.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024070441000000001, 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037986270000000002, 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224789+0000", 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909999999999997e-06 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224797+0000", 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224800+0000", 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.63e-06 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224844+0000", 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3661000000000001e-05 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224858+0000", 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3709e-05 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228528+0000", 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003670576 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228584+0000", 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5461999999999999e-05 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2534 3.f 3:f4c75701:::rbd_data.1152fda325c3.0000000000000006:head [stat,truncate 10038784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.228921+0000", 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01993489, 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002283485, 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:23.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2534 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228921+0000", 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228921+0000", 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228921+0000", 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0599999999999999e-07 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228925+0000", 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3730000000000001e-06 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228934+0000", 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4890000000000001e-06 2024-09-17T23:29:23.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228940+0000", 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3480000000000003e-06 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228981+0000", 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1009e-05 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:23.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.229032+0000", 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0949999999999998e-05 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.229104+0000", 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2900999999999999e-05 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.229833+0000", 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072821300000000002 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.231146+0000", 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001313449 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.231177+0000", 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0457e-05 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.231204+0000", 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7489999999999999e-05 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:23.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:23.363 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:23.364 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:23.361+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:23.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:447 punch from 0x24db800 to 0x24e2200, (0x6a00 bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:448 read 0x17dee00 thru 0x17eed21 (0xff22 bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:449 write 0x9456400 thru 0x9457047 (0xc48 bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:451 write 0x151b400 thru 0x151cafd (0x16fe bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:452 read 0x293b000 thru 0x294069c (0x569d bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:453 read 0x8723800 thru 0x87301a7 (0xc9a8 bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:454 read 0x4636e00 thru 0x4646a5a (0xfc5b bytes) 2024-09-17T23:29:23.366 INFO:teuthology.orchestra.run.smithi139.stdout:456 trunc from 0x9457200 to 0x61600 2024-09-17T23:29:23.374 INFO:teuthology.orchestra.run.smithi139.stdout:457 write 0x29efa00 thru 0x29ff7a3 (0xfda4 bytes) 2024-09-17T23:29:23.400 INFO:teuthology.orchestra.run.smithi139.stdout:460 write 0xe0ef000 thru 0xe0f9e29 (0xae2a bytes) 2024-09-17T23:29:23.401 INFO:teuthology.orchestra.run.smithi139.stdout:464 read 0x149b600 thru 0x149f94a (0x434b bytes) 2024-09-17T23:29:23.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:23.408 INFO:teuthology.orchestra.run.smithi139.stdout:465 trunc from 0xe0fa000 to 0x9719800 2024-09-17T23:29:23.422 INFO:teuthology.orchestra.run.smithi139.stdout:466 read 0x3ff1c00 thru 0x40008a0 (0xeca1 bytes) 2024-09-17T23:29:23.425 INFO:teuthology.orchestra.run.smithi139.stdout:467 read 0x782ac00 thru 0x78339ce (0x8dcf bytes) 2024-09-17T23:29:23.425 INFO:teuthology.orchestra.run.smithi139.stdout:469 write 0x484e800 thru 0x485d701 (0xef02 bytes) 2024-09-17T23:29:23.428 INFO:teuthology.orchestra.run.smithi139.stdout:470 trunc from 0x9719800 to 0xbb18400 2024-09-17T23:29:23.441 INFO:teuthology.orchestra.run.smithi139.stdout:471 read 0x42b1800 thru 0x42c0ed3 (0xf6d4 bytes) 2024-09-17T23:29:23.450 INFO:teuthology.orchestra.run.smithi139.stdout:476 trunc from 0xbb18400 to 0x43c6600 2024-09-17T23:29:23.465 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:23.462+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:23.467 INFO:teuthology.orchestra.run.smithi139.stdout:477 read 0x3900800 thru 0x39070ac (0x68ad bytes) 2024-09-17T23:29:23.469 INFO:teuthology.orchestra.run.smithi139.stdout:481 punch from 0xdf0a00 to 0xdf6200, (0x5800 bytes) 2024-09-17T23:29:23.469 INFO:teuthology.orchestra.run.smithi139.stdout:482 write 0x38a6800 thru 0x38b04e5 (0x9ce6 bytes) 2024-09-17T23:29:23.480 INFO:teuthology.orchestra.run.smithi139.stdout:483 trunc from 0x43c6600 to 0xcfb7c00 2024-09-17T23:29:23.493 INFO:teuthology.orchestra.run.smithi139.stdout:484 read 0xcf90400 thru 0xcf91128 (0xd29 bytes) 2024-09-17T23:29:23.494 INFO:teuthology.orchestra.run.smithi139.stdout:487 punch from 0x6de0600 to 0x6de7000, (0x6a00 bytes) 2024-09-17T23:29:23.495 INFO:teuthology.orchestra.run.smithi139.stdout:488 read 0xcc74400 thru 0xcc800a2 (0xbca3 bytes) 2024-09-17T23:29:23.496 INFO:teuthology.orchestra.run.smithi139.stdout:490 punch from 0x9e0e000 to 0x9e12400, (0x4400 bytes) 2024-09-17T23:29:23.496 INFO:teuthology.orchestra.run.smithi139.stdout:493 trunc from 0xcfb7c00 to 0xdbb6e00 2024-09-17T23:29:23.508 INFO:teuthology.orchestra.run.smithi139.stdout:494 read 0x8f41600 thru 0x8f42790 (0x1191 bytes) 2024-09-17T23:29:23.513 INFO:teuthology.orchestra.run.smithi139.stdout:495 trunc from 0xdbb6e00 to 0xa24e000 2024-09-17T23:29:23.523 INFO:teuthology.orchestra.run.smithi139.stdout:497 read 0x45cac00 thru 0x45d1f9c (0x739d bytes) 2024-09-17T23:29:23.525 INFO:teuthology.orchestra.run.smithi139.stdout:498 punch from 0x156aa00 to 0x1579000, (0xe600 bytes) 2024-09-17T23:29:23.526 INFO:teuthology.orchestra.run.smithi139.stdout:499 read 0x194f000 thru 0x195daeb (0xeaec bytes) 2024-09-17T23:29:23.526 INFO:teuthology.orchestra.run.smithi139.stdout:501 punch from 0x5ea2400 to 0x5eaa200, (0x7e00 bytes) 2024-09-17T23:29:23.526 INFO:teuthology.orchestra.run.smithi139.stdout:504 write 0x390e000 thru 0x39145d5 (0x65d6 bytes) 2024-09-17T23:29:23.529 INFO:teuthology.orchestra.run.smithi139.stdout:505 trunc from 0xa24e000 to 0x78d1c00 2024-09-17T23:29:23.533 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:23.533 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:23.533 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:23.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:23.546 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:23.549 INFO:teuthology.orchestra.run.smithi139.stdout:506 read 0x72aee00 thru 0x72b5656 (0x6857 bytes) 2024-09-17T23:29:23.563 INFO:teuthology.orchestra.run.smithi139.stdout:508 write 0x891c000 thru 0x89264d9 (0xa4da bytes) 2024-09-17T23:29:23.564 INFO:teuthology.orchestra.run.smithi139.stdout:511 punch from 0x4828200 to 0x4837600, (0xf400 bytes) 2024-09-17T23:29:23.565 INFO:teuthology.orchestra.run.smithi139.stdout:512 trunc from 0x8926600 to 0x8b72000 2024-09-17T23:29:23.566 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:23.563+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:23.568 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:23.577 INFO:teuthology.orchestra.run.smithi139.stdout:513 read 0x6b22e00 thru 0x6b24f3e (0x213f bytes) 2024-09-17T23:29:23.578 INFO:teuthology.orchestra.run.smithi139.stdout:516 write 0x427d800 thru 0x428528c (0x7a8d bytes) 2024-09-17T23:29:23.579 INFO:teuthology.orchestra.run.smithi139.stdout:518 punch from 0x6c23a00 to 0x6c2e000, (0xa600 bytes) 2024-09-17T23:29:23.579 INFO:teuthology.orchestra.run.smithi139.stdout:519 read 0x15c800 thru 0x15fb56 (0x3357 bytes) 2024-09-17T23:29:23.580 INFO:teuthology.orchestra.run.smithi139.stdout:520 read 0x5b59e00 thru 0x5b65eac (0xc0ad bytes) 2024-09-17T23:29:23.584 INFO:teuthology.orchestra.run.smithi139.stdout:525 trunc from 0x8b72000 to 0xc71d200 2024-09-17T23:29:23.590 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:23.590 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:23.597 INFO:teuthology.orchestra.run.smithi139.stdout:527 flatten 2024-09-17T23:29:23.608 INFO:teuthology.orchestra.run.smithi139.stdout:528 write 0xa5ea400 thru 0xa5edd79 (0x397a bytes) 2024-09-17T23:29:23.609 INFO:teuthology.orchestra.run.smithi139.stdout:530 clone 3 order 20 su 131072 sc 12 2024-09-17T23:29:23.611 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:23.611 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:23.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:23.628 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:23.645 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:23.645 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:23.662 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:23.662 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:23.666 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:23.665+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:23.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:23.767 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:23.766+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:23.819 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:23.820 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:23.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:23.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:29:23.873 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:23.868+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:23.970 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:23.968+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:23.985 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:23.985 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:23.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:24.070 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:24.067+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:24.146 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:24.146 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:24.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:24.199 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:24.197+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:24.264 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:24.265 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:24.265 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T23:29:24.272 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:24.271+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:24.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:24.373 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:24.370+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:24.398 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:24.398 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:24.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:24.493 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:24.490+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:24.575 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:24.573+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:24.585 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:24.585 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:24.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:24.676 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:24.674+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:24.748 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:24.748 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:24.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:24.800 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:24.797+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:24.864 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:24.864 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:24.869 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:29:24.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:24.915 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:24.912+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:24.979 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:24.976+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:25.034 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:25.034 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:25.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:25.081 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:25.079+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:25.181 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:25.179+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:25.189 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:25.189 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:25.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:25.210 INFO:teuthology.orchestra.run.smithi139.stdout:533 write 0x72b7200 thru 0x72c4c30 (0xda31 bytes) 2024-09-17T23:29:25.211 INFO:teuthology.orchestra.run.smithi139.stdout:535 punch from 0x62e5400 to 0x62f3e00, (0xea00 bytes) 2024-09-17T23:29:25.229 INFO:teuthology.orchestra.run.smithi139.stdout:538 write 0x45cb000 thru 0x45d96e0 (0xe6e1 bytes) 2024-09-17T23:29:25.230 INFO:teuthology.orchestra.run.smithi139.stdout:539 write 0xb157400 thru 0xb15bdfb (0x49fc bytes) 2024-09-17T23:29:25.230 INFO:teuthology.orchestra.run.smithi139.stdout:545 read 0x21f600 thru 0x22ed37 (0xf738 bytes) 2024-09-17T23:29:25.231 INFO:teuthology.orchestra.run.smithi139.stdout:549 read 0x7d48800 thru 0x7d53e8c (0xb68d bytes) 2024-09-17T23:29:25.232 INFO:teuthology.orchestra.run.smithi139.stdout:550 write 0x14ae00 thru 0x155233 (0xa434 bytes) 2024-09-17T23:29:25.232 INFO:teuthology.orchestra.run.smithi139.stdout:551 punch from 0x7b34a00 to 0x7b36200, (0x1800 bytes) 2024-09-17T23:29:25.244 INFO:teuthology.orchestra.run.smithi139.stdout:555 punch from 0x4e00200 to 0x4e05600, (0x5400 bytes) 2024-09-17T23:29:25.249 INFO:teuthology.orchestra.run.smithi139.stdout:557 read 0x56a4800 thru 0x56b1637 (0xce38 bytes) 2024-09-17T23:29:25.249 INFO:teuthology.orchestra.run.smithi139.stdout:558 read 0xac27e00 thru 0xac35fbe (0xe1bf bytes) 2024-09-17T23:29:25.249 INFO:teuthology.orchestra.run.smithi139.stdout:559 read 0x30e600 thru 0x31ddf1 (0xf7f2 bytes) 2024-09-17T23:29:25.249 INFO:teuthology.orchestra.run.smithi139.stdout:560 read 0x98a7e00 thru 0x98b69f8 (0xebf9 bytes) 2024-09-17T23:29:25.257 INFO:teuthology.orchestra.run.smithi139.stdout:561 trunc from 0xc71d200 to 0x655c600 2024-09-17T23:29:25.282 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:25.280+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:25.300 INFO:teuthology.orchestra.run.smithi139.stdout:564 trunc from 0x655c600 to 0x106c400 2024-09-17T23:29:25.328 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:25.329 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:25.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:25.350 INFO:teuthology.orchestra.run.smithi139.stdout:565 write 0x4227c00 thru 0x4236ea5 (0xf2a6 bytes) 2024-09-17T23:29:25.351 INFO:teuthology.orchestra.run.smithi139.stdout:566 read 0x1cd3000 thru 0x1cd58a0 (0x28a1 bytes) 2024-09-17T23:29:25.351 INFO:teuthology.orchestra.run.smithi139.stdout:569 read 0x413c200 thru 0x4148195 (0xbf96 bytes) 2024-09-17T23:29:25.354 INFO:teuthology.orchestra.run.smithi139.stdout:572 read 0x4acc00 thru 0x4b9fcf (0xd3d0 bytes) 2024-09-17T23:29:25.354 INFO:teuthology.orchestra.run.smithi139.stdout:573 punch from 0x1dae000 to 0x1db3200, (0x5200 bytes) 2024-09-17T23:29:25.372 INFO:teuthology.orchestra.run.smithi139.stdout:574 write 0x8c90400 thru 0x8c949cb (0x45cc bytes) 2024-09-17T23:29:25.377 INFO:teuthology.orchestra.run.smithi139.stdout:577 punch from 0x6f6c000 to 0x6f70600, (0x4600 bytes) 2024-09-17T23:29:25.383 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:25.382+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:25.385 INFO:teuthology.orchestra.run.smithi139.stdout:578 trunc from 0x8c94a00 to 0xe7a2600 2024-09-17T23:29:25.410 INFO:teuthology.orchestra.run.smithi139.stdout:579 trunc from 0xe7a2600 to 0x306f200 2024-09-17T23:29:25.430 INFO:teuthology.orchestra.run.smithi139.stdout:580 read 0x1c54200 thru 0x1c5cbe6 (0x89e7 bytes) 2024-09-17T23:29:25.436 INFO:teuthology.orchestra.run.smithi139.stdout:581 clone 4 order 24 su 4194304 sc 6 2024-09-17T23:29:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:25.457 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:25.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:25.510 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:25.507+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:25.573 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:25.573 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:25.580 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:25.580 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:25.581 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-17T23:29:25.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T23:29:25.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:25.623 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:25.621+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:25.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:25.693 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:25.694 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:25.694 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:25.692+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:25.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:25.786 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:25.784+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:25.795 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:25.795 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:25.813 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:25.813 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:25.829 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:25.830 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:25.847 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:25.847 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:25.854 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:25.854 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:25.854 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:25.854 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:25.854 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:25.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:25.864 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:25.864 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:25.884 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:25.884 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:25.887 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:25.884+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:25.907 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:29:25.905+0000 7f21c94c5640 -1 mon.a@0(leader).osd e29 definitely_dead 0 2024-09-17T23:29:25.950 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone3 from 0x306f200 (overlap 0x106c400) to 0x0 2024-09-17T23:29:25.980 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:25.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.475440869, 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046920809999999999, 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:25.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502234+0000", 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9400000000000001e-06 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502243+0000", 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4290000000000007e-06 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:25.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502260+0000", 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6872e-05 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502272+0000", 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1792e-05 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502599+0000", 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032732099999999999 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:25.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502674+0000", 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.436e-05 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502795+0000", 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121549 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.504847+0000", 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020513620000000002 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506847+0000", 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020007689999999999 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506887+0000", 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9647000000000002e-05 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506924+0000", 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7039999999999998e-05 2024-09-17T23:29:25.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:175 3.b e23/19 3:d3468d19:::rbd_data.1152f108f2b3.0000000000000009:head v 23'2, mlcod=0'0)", 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1203237760000002, 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031410980000000002, 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:25.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:25.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857350+0000", 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857354+0000", 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1520000000000002e-06 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857361+0000", 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8299999999999996e-06 2024-09-17T23:29:25.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857367+0000", 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857408+0000", 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1065000000000001e-05 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857425+0000", 2024-09-17T23:29:25.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6977999999999999e-05 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.860455+0000", 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030295080000000002 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.860490+0000", 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5401999999999997e-05 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:355 3.9 3:91b1f5bf:::rbd_data.1152f108f2b3.0000000000000029:head [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.9641915560000003, 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029455660000000002, 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 355 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:25.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013482+0000", 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0529999999999999e-06 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013494+0000", 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1239e-05 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:25.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013508+0000", 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4321000000000001e-05 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013518+0000", 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8320000000000001e-06 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013564+0000", 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6347999999999997e-05 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013617+0000", 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2679000000000002e-05 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.016427+0000", 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002810094 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:25.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4468.0:910 3.f 3:f7e9d27a:::rbd_object_map.11522ab9a303:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5500868260000003, 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003778319, 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4468", 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 910 2024-09-17T23:29:25.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:25.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427587+0000", 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999998e-07 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427591+0000", 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9160000000000003e-06 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427600+0000", 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5740000000000002e-06 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:25.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427606+0000", 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9220000000000002e-06 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427654+0000", 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8059999999999997e-05 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427748+0000", 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3789999999999995e-05 2024-09-17T23:29:26.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.430511+0000", 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027624400000000001 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431143+0000", 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063282300000000002 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431309+0000", 2024-09-17T23:29:26.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016585999999999999 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431340+0000", 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0731000000000001e-05 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431364+0000", 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.44e-05 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2049 3.a 3:52cea45b:::rbd_data.1152fda325c3.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.8827657650000003, 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002789297, 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2049 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094908+0000", 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094911+0000", 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-06 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094919+0000", 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8009999999999993e-06 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094924+0000", 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4469999999999999e-06 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094967+0000", 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3343000000000002e-05 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.095505+0000", 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053770000000000001 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.095622+0000", 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117087 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.096354+0000", 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073242800000000003 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097606+0000", 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001251432 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097660+0000", 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3693000000000002e-05 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097696+0000", 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6807000000000002e-05 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2378 3.c 3:36bb075d:::rbd_header.1152fda325c3:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.8239368250000001, 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024627289999999999, 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2378 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153737+0000", 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5600000000000005e-07 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153740+0000", 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0319999999999999e-06 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153746+0000", 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153750+0000", 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6710000000000001e-06 2024-09-17T23:29:26.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153784+0000", 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4799999999999999e-05 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153830+0000", 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5719000000000001e-05 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153947+0000", 2024-09-17T23:29:26.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011649100000000001 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.154698+0000", 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075154900000000001 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156144+0000", 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445794 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156173+0000", 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9436000000000001e-05 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156199+0000", 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-05 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2477 3.17 e26/19 3:eee396ce:::rbd_object_map.1152fda325c3:head v 26'40, mlcod=26'38)", 2024-09-17T23:29:26.014 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.784166801, 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034184390000000001, 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193514+0000", 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193521+0000", 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1230000000000002e-06 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193526+0000", 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1780000000000002e-06 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193596+0000", 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9776000000000004e-05 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193618+0000", 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2603e-05 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196879+0000", 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032604230000000001 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196924+0000", 2024-09-17T23:29:26.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5816000000000003e-05 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2478 3.12 e26/19 3:4952c6c2:::rbd_data.1152fda325c3.0000000000000000:head v 26'9, mlcod=26'7)", 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7840508449999999, 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032918359999999998, 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193652+0000", 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9592000000000001e-05 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193658+0000", 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9669999999999998e-06 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193662+0000", 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6959999999999998e-06 2024-09-17T23:29:26.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193711+0000", 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9685000000000003e-05 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193724+0000", 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2999999999999999e-05 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196886+0000", 2024-09-17T23:29:26.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031617390000000002 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196914+0000", 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7724000000000001e-05 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2529 3.15 3:af4fb69a:::rbd_data.1152fda325c3.0000000000000001:head [stat,truncate 10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7545125690000001, 2024-09-17T23:29:26.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054525839999999999, 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2529 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223161+0000", 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7299999999999996e-07 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223164+0000", 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0709999999999999e-06 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223172+0000", 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0450000000000001e-06 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223176+0000", 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2950000000000004e-06 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223233+0000", 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6994999999999999e-05 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223318+0000", 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4400999999999994e-05 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223467+0000", 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149796 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224157+0000", 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000689465 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228516+0000", 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004358682 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228573+0000", 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7596999999999998e-05 2024-09-17T23:29:26.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228613+0000", 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9663999999999998e-05 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2531 3.17 e26/19 3:ec02a8b5:::rbd_data.1152fda325c3.0000000000000003:head v 26'43, mlcod=26'41)", 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7542277840000002, 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051349280000000004, 2024-09-17T23:29:26.029 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223451+0000", 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223458+0000", 2024-09-17T23:29:26.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4150000000000004e-06 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223461+0000", 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3390000000000001e-06 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223501+0000", 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0201999999999999e-05 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223521+0000", 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0085000000000002e-05 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228525+0000", 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050040179999999998 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228580+0000", 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4688999999999998e-05 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2532 3.1b e26/19 3:d90b4fac:::rbd_data.1152fda325c3.0000000000000004:head v 26'13, mlcod=26'11)", 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7528878109999999, 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037986270000000002, 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:26.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224789+0000", 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909999999999997e-06 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224797+0000", 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224800+0000", 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.63e-06 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224844+0000", 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3661000000000001e-05 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224858+0000", 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3709e-05 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228528+0000", 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003670576 2024-09-17T23:29:26.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228584+0000", 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5461999999999999e-05 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2584 3.f 3:f4c75701:::rbd_data.1152fda325c3.0000000000000006:head [stat,truncate 6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7280452679999998, 2024-09-17T23:29:26.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041927789999999998, 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2584 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:26.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249631+0000", 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9059999999999998e-06 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249644+0000", 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2934e-05 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249648+0000", 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249690+0000", 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999998e-05 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249765+0000", 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4518999999999996e-05 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249913+0000", 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001483 2024-09-17T23:29:26.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253747+0000", 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003834306 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253767+0000", 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0047e-05 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253798+0000", 2024-09-17T23:29:26.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1090999999999998e-05 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253820+0000", 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1963999999999998e-05 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2577 3.12 e26/19 3:4952c6c2:::rbd_data.1152fda325c3.0000000000000000:head v 26'12, mlcod=26'10)", 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.250165+0000", 2024-09-17T23:29:26.044 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7275074740000003, 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035971060000000001, 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250165+0000", 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250165+0000", 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250166+0000", 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2399999999999997e-07 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250180+0000", 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3444999999999999e-05 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250190+0000", 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0633e-05 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250197+0000", 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1559999999999998e-06 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250244+0000", 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6922000000000002e-05 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250265+0000", 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0797000000000001e-05 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253731+0000", 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00346605 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253762+0000", 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1279000000000003e-05 2024-09-17T23:29:26.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2580 3.17 e26/19 3:ec02a8b5:::rbd_data.1152fda325c3.0000000000000003:head v 26'45, mlcod=26'43)", 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7274583140000002, 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036269259999999999, 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250219+0000", 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6849999999999999e-06 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250225+0000", 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250230+0000", 2024-09-17T23:29:26.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250287+0000", 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6539000000000002e-05 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250304+0000", 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7439e-05 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253824+0000", 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035195550000000002 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253841+0000", 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7793999999999999e-05 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2581 3.1b e26/19 3:d90b4fac:::rbd_data.1152fda325c3.0000000000000004:head v 26'14, mlcod=26'12)", 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:26.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7274239219999998, 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035399889999999999, 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:26.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1799999999999995e-07 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250258+0000", 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0870000000000007e-06 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250267+0000", 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2240000000000001e-06 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250272+0000", 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6269999999999997e-06 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250324+0000", 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1838000000000001e-05 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250342+0000", 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7537000000000002e-05 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253749+0000", 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003407546 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253789+0000", 2024-09-17T23:29:26.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9612e-05 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2690 3.a 3:52cea45b:::rbd_data.1152fda325c3.0000000000000008:head [set-alloc-hint object_size 16777216 write_size 16777216,write 2974720~3584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.296602+0000", 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.6810707090000001, 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00246517, 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2690 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296602+0000", 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296602+0000", 2024-09-17T23:29:26.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296603+0000", 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0699999999999996e-07 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296607+0000", 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3780000000000003e-06 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296617+0000", 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3610000000000007e-06 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296621+0000", 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8300000000000003e-06 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296674+0000", 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2225999999999998e-05 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296751+0000", 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7873999999999993e-05 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296898+0000", 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014618399999999999 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.298802+0000", 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904053 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.298972+0000", 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000170318 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.299028+0000", 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6289999999999998e-05 2024-09-17T23:29:26.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.299067+0000", 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8949000000000003e-05 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2703 3.17 e26/19 3:eee396ce:::rbd_object_map.1152fda325c3:head v 26'54, mlcod=26'52)", 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.322049+0000", 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.655624016, 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003613434, 2024-09-17T23:29:26.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322049+0000", 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322049+0000", 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322050+0000", 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5570000000000001e-06 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322061+0000", 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0672999999999999e-05 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322086+0000", 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.478e-05 2024-09-17T23:29:26.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322098+0000", 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2186e-05 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322203+0000", 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105099 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322237+0000", 2024-09-17T23:29:26.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3716e-05 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.325527+0000", 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032903479999999998 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.325662+0000", 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013507499999999999 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4551.0:758 3.e 3:76bd1ce5:::rbd_object_map.1152846a29a0.0000000000000006:head [writefull 0~47] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:25.871468+0000", 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.10620481, 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042291869999999997, 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4551", 2024-09-17T23:29:26.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 758 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871468+0000", 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871468+0000", 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871470+0000", 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3580000000000001e-06 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871486+0000", 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5542000000000001e-05 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871508+0000", 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1999999999999999e-05 2024-09-17T23:29:26.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871520+0000", 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1858999999999999e-05 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871639+0000", 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119356 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871644+0000", 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.874231+0000", 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025869880000000001 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.874363+0000", 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131667 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.874508+0000", 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014457399999999999 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875352+0000", 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084419200000000001 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875597+0000", 2024-09-17T23:29:26.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024543600000000003 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875653+0000", 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6178000000000003e-05 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875697+0000", 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3890999999999997e-05 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4551.0:804 3.3 3:c527221d:::rbd_data.1152846a29a0.000000000000000b:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:25.954264+0000", 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.023408333999999999, 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00297993, 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4551", 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:26.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 804 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954264+0000", 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954264+0000", 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954266+0000", 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.124e-06 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954270+0000", 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5630000000000002e-06 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954281+0000", 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1082e-05 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954287+0000", 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.716e-06 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954353+0000", 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5656999999999998e-05 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954477+0000", 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124468 2024-09-17T23:29:26.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.957244+0000", 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027673200000000002 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4551.0:826 3.3 3:c4f02a04:::rbd_data.1152846a29a0.0000000000000021:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:25.955136+0000", 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.022536398999999999, 2024-09-17T23:29:26.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041672549999999999, 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4551", 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 826 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955136+0000", 2024-09-17T23:29:26.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955136+0000", 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955138+0000", 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.156e-06 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955143+0000", 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2050000000000001e-06 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955167+0000", 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.457e-05 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955180+0000", 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2588e-05 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955248+0000", 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7714999999999998e-05 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955321+0000", 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955489+0000", 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167746 2024-09-17T23:29:26.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.957310+0000", 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018216910000000001 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.959209+0000", 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001898336 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.959264+0000", 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5766000000000001e-05 2024-09-17T23:29:26.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.959304+0000", 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9239000000000002e-05 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.086 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:26.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:26.088 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:26.086+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:26.188 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:26.187+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:26.237 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T23:29:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:26.238 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:26.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:26.290 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:26.286+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:26.391 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:26.388+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:26.398 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:26.398 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:26.398 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:26.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:26.492 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:26.489+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:26.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.047580700000001, 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:26.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:26.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:26.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:26.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:26.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:26.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1858 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.506497413, 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010693322999999999, 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1858 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.22e-07 2024-09-17T23:29:26.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012524+0000", 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1209999999999999e-06 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012527+0000", 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2490000000000001e-06 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012530+0000", 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5579999999999999e-06 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023039+0000", 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010508671000000001 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023116+0000", 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7574e-05 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023216+0000", 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9927999999999996e-05 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1859 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5064510330000003, 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010887203, 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1859 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012570+0000", 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012571+0000", 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.75e-06 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012578+0000", 2024-09-17T23:29:26.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012581+0000", 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023262+0000", 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010680640999999999 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023344+0000", 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2785000000000001e-05 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023456+0000", 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111983 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1861 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5064349139999997, 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011093442, 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1861 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012586+0000", 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012590+0000", 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.724e-06 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012594+0000", 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6320000000000001e-06 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012597+0000", 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6769999999999999e-06 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023495+0000", 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010898309 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023580+0000", 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023679+0000", 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9230999999999999e-05 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1862 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5064201390000003, 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011325251, 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1862 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0600000000000001e-07 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012602+0000", 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2589999999999999e-06 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012605+0000", 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.072e-06 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012607+0000", 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023725+0000", 2024-09-17T23:29:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01111767 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023810+0000", 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840999999999996e-05 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023925+0000", 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011567500000000001 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1863 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.506410061, 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011535393, 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1863 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-07 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012612+0000", 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1799999999999999e-06 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012615+0000", 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012617+0000", 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5000000000000002e-06 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023967+0000", 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011349704 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024045+0000", 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7955999999999997e-05 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024146+0000", 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100691 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1864 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 9830400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063999940000001, 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01177426, 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1864 2024-09-17T23:29:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1400000000000001e-07 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012622+0000", 2024-09-17T23:29:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-06 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012625+0000", 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.128e-06 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012627+0000", 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024192+0000", 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011564606999999999 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024282+0000", 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9752000000000004e-05 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024395+0000", 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011288199999999999 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1865 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063898920000001, 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011984761999999999, 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1865 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012631+0000", 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012632+0000", 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.215e-06 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012635+0000", 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1489999999999998e-06 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012637+0000", 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.57e-06 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024435+0000", 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011797368000000001 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024512+0000", 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7081000000000003e-05 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024615+0000", 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103176 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1866 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063798429999999, 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012220981000000001, 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1866 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012641+0000", 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:29:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012642+0000", 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1230000000000001e-06 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012645+0000", 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.18e-06 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012647+0000", 2024-09-17T23:29:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4329999999999998e-06 2024-09-17T23:29:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024660+0000", 2024-09-17T23:29:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012012372 2024-09-17T23:29:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024747+0000", 2024-09-17T23:29:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7374999999999996e-05 2024-09-17T23:29:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024861+0000", 2024-09-17T23:29:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011429000000000001 2024-09-17T23:29:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1867 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063697810000001, 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012428603999999999, 2024-09-17T23:29:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1867 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012651+0000", 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012652+0000", 2024-09-17T23:29:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1969999999999999e-06 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012655+0000", 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1530000000000001e-06 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012658+0000", 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.543e-06 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024903+0000", 2024-09-17T23:29:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012244994 2024-09-17T23:29:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024977+0000", 2024-09-17T23:29:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3951999999999999e-05 2024-09-17T23:29:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025079+0000", 2024-09-17T23:29:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102546 2024-09-17T23:29:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1868 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063579049999998, 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012658264000000001, 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1868 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012663+0000", 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012664+0000", 2024-09-17T23:29:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-09-17T23:29:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012667+0000", 2024-09-17T23:29:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1839999999999999e-06 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012669+0000", 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4990000000000001e-06 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025123+0000", 2024-09-17T23:29:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012453138000000001 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025207+0000", 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4153000000000006e-05 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025321+0000", 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113874 2024-09-17T23:29:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1869 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063476900000001, 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012863466, 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1869 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012674+0000", 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.111e-06 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012677+0000", 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1190000000000001e-06 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012680+0000", 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5809999999999999e-06 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025360+0000", 2024-09-17T23:29:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012680593 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025436+0000", 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5768999999999999e-05 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025536+0000", 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100068 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1870 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063376010000002, 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013108685, 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1870 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012684+0000", 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0890000000000001e-06 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012687+0000", 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.146e-06 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012690+0000", 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5519999999999999e-06 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025589+0000", 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012899713 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025676+0000", 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6457000000000002e-05 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025791+0000", 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115516 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1871 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063275149999997, 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013256860000000001, 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1871 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012695+0000", 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.855e-06 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012698+0000", 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2600000000000001e-06 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012701+0000", 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025836+0000", 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013135338999999999 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025884+0000", 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7846999999999998e-05 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025950+0000", 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5791999999999996e-05 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1872 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063163899999998, 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013498906, 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1872 2024-09-17T23:29:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012705+0000", 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2619999999999999e-06 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012709+0000", 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1990000000000002e-06 2024-09-17T23:29:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012711+0000", 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4310000000000001e-06 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025996+0000", 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013284921 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026084+0000", 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7754e-05 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026203+0000", 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119133 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1873 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5063061920000003, 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013625369999999999, 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1873 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-07 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012723+0000", 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6079999999999996e-06 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012727+0000", 2024-09-17T23:29:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0319999999999997e-06 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012730+0000", 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7590000000000002e-06 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026236+0000", 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013505980000000001 2024-09-17T23:29:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026279+0000", 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3838000000000003e-05 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026339+0000", 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9953e-05 2024-09-17T23:29:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1874 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 15073280~31744] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5062862670000001, 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013877722, 2024-09-17T23:29:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1874 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4699999999999998e-07 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012736+0000", 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3740000000000001e-06 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012739+0000", 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3019999999999999e-06 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012741+0000", 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5739999999999998e-06 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026384+0000", 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013642965 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026465+0000", 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026612+0000", 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014634700000000001 2024-09-17T23:29:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1875 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5062756500000001, 2024-09-17T23:29:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013997559, 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1875 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-07 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012746+0000", 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2729999999999999e-06 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012749+0000", 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012752+0000", 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.672e-06 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026646+0000", 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013894098000000001 2024-09-17T23:29:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026687+0000", 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0540000000000001e-05 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026742+0000", 2024-09-17T23:29:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5585999999999999e-05 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1876 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.50626521, 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014222617999999999, 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1876 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3300000000000001e-07 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012757+0000", 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.353e-06 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012760+0000", 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1930000000000002e-06 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012762+0000", 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026784+0000", 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014021909000000001 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026867+0000", 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3014000000000002e-05 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026978+0000", 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110282 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1877 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5062283920000001, 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014306071, 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1877 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3799999999999998e-07 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012794+0000", 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4169999999999999e-06 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012800+0000", 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.896e-06 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012803+0000", 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2760000000000001e-06 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027001+0000", 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014198555999999999 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027040+0000", 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8479000000000002e-05 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027098+0000", 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8109000000000002e-05 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:26.645 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:26.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:26.647 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:26.645+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:26.748 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:26.746+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:26.795 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:26.795 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:26.795 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:26.798 INFO:teuthology.orchestra.run.smithi139.stdout:582 write 0x2d4e400 thru 0x2d52da5 (0x49a6 bytes) 2024-09-17T23:29:26.798 INFO:teuthology.orchestra.run.smithi139.stdout:583 punch from 0x1bdfa00 to 0x1bed200, (0xd800 bytes) 2024-09-17T23:29:26.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:26.827 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-17T23:29:26.828 INFO:teuthology.orchestra.run.smithi139.stdout:588 read 0x296fc00 thru 0x2973901 (0x3d02 bytes) 2024-09-17T23:29:26.829 INFO:teuthology.orchestra.run.smithi139.stdout:590 trunc from 0x306f200 to 0x3ecf400 2024-09-17T23:29:26.843 INFO:teuthology.orchestra.run.smithi139.stdout:591 read 0x1bfc200 thru 0x1c07220 (0xb021 bytes) 2024-09-17T23:29:26.843 INFO:teuthology.orchestra.run.smithi139.stdout:592 read 0x15c9400 thru 0x15d41ae (0xadaf bytes) 2024-09-17T23:29:26.845 INFO:teuthology.orchestra.run.smithi139.stdout:593 trunc from 0x3ecf400 to 0x353c200 2024-09-17T23:29:26.868 INFO:teuthology.orchestra.run.smithi139.stdout:596 read 0x20f4200 thru 0x20f6184 (0x1f85 bytes) 2024-09-17T23:29:26.868 INFO:teuthology.orchestra.run.smithi139.stdout:597 punch from 0x3273400 to 0x327fa00, (0xc600 bytes) 2024-09-17T23:29:26.869 INFO:teuthology.orchestra.run.smithi139.stdout:599 trunc from 0x353c200 to 0x4835c00 2024-09-17T23:29:26.875 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.2. 2024-09-17T23:29:26.875 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:26.873+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:26.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:29:26.895 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-17T23:29:26.950 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:26.947+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:26.958 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:26.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:27.050 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:27.048+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:27.078 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:27.078 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:27.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:27.078 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:27.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4236.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-17T23:29:27.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:27.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.574805077000001, 2024-09-17T23:29:27.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003809191, 2024-09-17T23:29:27.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:27.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:27.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-17T23:29:27.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499873+0000", 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.91e-06 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499897+0000", 2024-09-17T23:29:27.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4607999999999999e-05 2024-09-17T23:29:27.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499912+0000", 2024-09-17T23:29:27.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.435e-05 2024-09-17T23:29:27.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499975+0000", 2024-09-17T23:29:27.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2942999999999997e-05 2024-09-17T23:29:27.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.500010+0000", 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4839000000000003e-05 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503630+0000", 2024-09-17T23:29:27.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003619911 2024-09-17T23:29:27.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503672+0000", 2024-09-17T23:29:27.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2315999999999999e-05 2024-09-17T23:29:27.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:12 3.1 e22/19 3:86f27224:::rbd_header.1152f108f2b3:head v 22'3, mlcod=22'1)", 2024-09-17T23:29:27.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:17.019038+0000", 2024-09-17T23:29:27.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.055629511999999, 2024-09-17T23:29:27.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000825269, 2024-09-17T23:29:27.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019038+0000", 2024-09-17T23:29:27.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019038+0000", 2024-09-17T23:29:27.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019040+0000", 2024-09-17T23:29:27.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2730000000000001e-06 2024-09-17T23:29:27.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019057+0000", 2024-09-17T23:29:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6500000000000001e-05 2024-09-17T23:29:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019086+0000", 2024-09-17T23:29:27.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8866000000000001e-05 2024-09-17T23:29:27.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019101+0000", 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5489999999999999e-05 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019214+0000", 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011244200000000001 2024-09-17T23:29:27.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019256+0000", 2024-09-17T23:29:27.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2182999999999999e-05 2024-09-17T23:29:27.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019832+0000", 2024-09-17T23:29:27.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00057584499999999996 2024-09-17T23:29:27.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:17.019863+0000", 2024-09-17T23:29:27.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.167e-05 2024-09-17T23:29:27.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:16 3.1 e22/19 3:86f27224:::rbd_header.1152f108f2b3:head v 22'4, mlcod=22'2)", 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.680943+0000", 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3937244769999992, 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00077256600000000003, 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.680943+0000", 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.680943+0000", 2024-09-17T23:29:27.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.680945+0000", 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6109999999999999e-06 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.680968+0000", 2024-09-17T23:29:27.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3156e-05 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.680983+0000", 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262999999999999e-05 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.680992+0000", 2024-09-17T23:29:27.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7739999999999993e-06 2024-09-17T23:29:27.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.681052+0000", 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9996e-05 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.681071+0000", 2024-09-17T23:29:27.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9315999999999999e-05 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.681679+0000", 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060804800000000003 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.681716+0000", 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6402000000000001e-05 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:38 3.1 e23/19 3:86f27224:::rbd_header.1152f108f2b3:head v 23'7, mlcod=23'5)", 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.699466+0000", 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3752016579999999, 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065594000000000004, 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699466+0000", 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699466+0000", 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699467+0000", 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699472+0000", 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-17T23:29:27.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699481+0000", 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5189999999999998e-06 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699486+0000", 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.904e-06 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699529+0000", 2024-09-17T23:29:27.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2237000000000002e-05 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.699542+0000", 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.379e-05 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.700090+0000", 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054766099999999996 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.700122+0000", 2024-09-17T23:29:27.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2023999999999997e-05 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.1 epoch 23/19 rep_tid 10 entries 23'8 (0'0) error 3:86f27224:::rbd_header.1152f108f2b3:head by client.4434.0:52 0.000000 -16 [r=-16+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 23'6)", 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.710219+0000", 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3644487410000004, 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068903900000000001, 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:27.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710219+0000", 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710219+0000", 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710220+0000", 2024-09-17T23:29:27.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4699999999999999e-06 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710236+0000", 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5153e-05 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710275+0000", 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9749000000000002e-05 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710299+0000", 2024-09-17T23:29:27.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3810000000000001e-05 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710451+0000", 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015207400000000001 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.710908+0000", 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00045678299999999999 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:98 3.1d 3:b8a70351:::rbd_data.1152f108f2b3.0000000000000039:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1884160~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.759663+0000", 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3150047520000001, 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016012240000000001, 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:27.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 98 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759663+0000", 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759663+0000", 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759664+0000", 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4219999999999999e-06 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759742+0000", 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7138000000000006e-05 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759750+0000", 2024-09-17T23:29:27.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1859999999999992e-06 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759761+0000", 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0959e-05 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759810+0000", 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710999999999999e-05 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759900+0000", 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9531999999999996e-05 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.759987+0000", 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7022999999999997e-05 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.760912+0000", 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092462800000000004 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.761201+0000", 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028985600000000001 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.761236+0000", 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4613000000000002e-05 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.761264+0000", 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8155999999999999e-05 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:104 3.1d 3:b8a70351:::rbd_data.1152f108f2b3.0000000000000039:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.767663+0000", 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.30700517, 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087967500000000001, 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:27.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 104 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767663+0000", 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767663+0000", 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767663+0000", 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.16e-07 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767667+0000", 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5769999999999998e-06 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767679+0000", 2024-09-17T23:29:27.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2289999999999999e-05 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767687+0000", 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0760000000000003e-06 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767724+0000", 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6137999999999999e-05 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767778+0000", 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4871999999999999e-05 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.767846+0000", 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7759e-05 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.768286+0000", 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000439889 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.768472+0000", 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000186112 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.768505+0000", 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2366999999999997e-05 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.768542+0000", 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7679000000000003e-05 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:129 3.0 e23/19 3:028fa2d0:::rbd_data.1152f108f2b3.0000000000000027:head v 23'1, mlcod=0'0)", 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.787223+0000", 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.2874444740000008, 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001021611, 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787223+0000", 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787223+0000", 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787224+0000", 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787260+0000", 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5809000000000002e-05 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787281+0000", 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0907000000000002e-05 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787292+0000", 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1786e-05 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787395+0000", 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010239900000000001 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.787427+0000", 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2419000000000003e-05 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.788200+0000", 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000773342 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.788245+0000", 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4422000000000003e-05 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:216 3.e 3:7303705d:::rbd_data.1152f108f2b3.000000000000001d:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2662400~59904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.901792+0000", 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1728759770000003, 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001467178, 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 216 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901792+0000", 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901792+0000", 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901793+0000", 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901884+0000", 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1514999999999994e-05 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901896+0000", 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1394e-05 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901901+0000", 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1590000000000001e-06 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.901958+0000", 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7229999999999999e-05 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.902063+0000", 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104603 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.902146+0000", 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3106e-05 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.902938+0000", 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079247300000000003 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.903205+0000", 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026623899999999998 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.903233+0000", 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8560000000000001e-05 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.903259+0000", 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5707999999999998e-05 2024-09-17T23:29:27.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.1 epoch 23/19 rep_tid 50 entries 23'26 (0'0) error 3:86f27224:::rbd_header.1152f108f2b3:head by client.4434.0:149 0.000000 -16 [r=-16+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 23'24)", 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.904442+0000", 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1702253999999996, 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001158168, 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904442+0000", 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904442+0000", 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904443+0000", 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3399999999999999e-07 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904449+0000", 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3679999999999998e-06 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904463+0000", 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4068e-05 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904471+0000", 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0029999999999994e-06 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.904525+0000", 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3372000000000003e-05 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.905600+0000", 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075723 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:238 3.14 e23/19 3:2ba1420b:::rbd_data.1152f108f2b3.000000000000000a:head v 23'1, mlcod=0'0)", 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.926497+0000", 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1481704910000001, 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079107799999999996, 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926497+0000", 2024-09-17T23:29:27.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926497+0000", 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926498+0000", 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4199999999999995e-07 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926527+0000", 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8806000000000002e-05 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926537+0000", 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926542+0000", 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4109999999999998e-06 2024-09-17T23:29:27.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926586+0000", 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4394999999999998e-05 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.926598+0000", 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2244e-05 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.927261+0000", 2024-09-17T23:29:27.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000662796 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.927288+0000", 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7243999999999999e-05 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "pg_update_log_missing(3.1 epoch 23/19 rep_tid 59 entries 23'30 (0'0) error 3:86f27224:::rbd_header.1152f108f2b3:head by client.4447.0:253 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 23'28)", 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.939049+0000", 2024-09-17T23:29:27.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1356182530000005, 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069104699999999997, 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939049+0000", 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939049+0000", 2024-09-17T23:29:27.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939051+0000", 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939071+0000", 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9919999999999999e-05 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939111+0000", 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0009000000000003e-05 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939134+0000", 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2719999999999999e-05 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939308+0000", 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017365300000000001 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.939741+0000", 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00043281500000000002 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:258 3.1 e23/19 3:85745777:::rbd_data.1152f108f2b3.0000000000000013:head v 23'31, mlcod=23'29)", 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.946388+0000", 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1282802180000004, 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001041014, 2024-09-17T23:29:27.152 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946388+0000", 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946388+0000", 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946388+0000", 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0399999999999995e-07 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946444+0000", 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5624999999999997e-05 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946454+0000", 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0373999999999999e-05 2024-09-17T23:29:27.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946460+0000", 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0950000000000001e-06 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946513+0000", 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2454999999999998e-05 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.946536+0000", 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3604e-05 2024-09-17T23:29:27.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.947397+0000", 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00086036200000000004 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.947429+0000", 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1795000000000003e-05 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:282 3.1 e23/19 3:86f27224:::rbd_header.1152f108f2b3:head v 23'35, mlcod=23'33)", 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.969520+0000", 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1051475039999996, 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075307599999999998, 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969520+0000", 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969520+0000", 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969521+0000", 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0699999999999996e-07 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969530+0000", 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969543+0000", 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3346000000000001e-05 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969550+0000", 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969613+0000", 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2849999999999996e-05 2024-09-17T23:29:27.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.969628+0000", 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4905999999999999e-05 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.970243+0000", 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061456599999999998 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.970273+0000", 2024-09-17T23:29:27.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0704000000000003e-05 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:353 3.14 e23/19 3:2ba1420b:::rbd_data.1152f108f2b3.000000000000000a:head v 23'2, mlcod=0'0)", 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.010978+0000", 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0636895919999994, 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071726600000000004, 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.010978+0000", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.010978+0000", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.010979+0000", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999997e-07 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.010991+0000", 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1664999999999999e-05 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.011003+0000", 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2362e-05 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.011009+0000", 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5979999999999996e-06 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.011063+0000", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4085999999999997e-05 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.011077+0000", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4476e-05 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.011663+0000", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058542999999999996 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.011695+0000", 2024-09-17T23:29:27.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2793999999999998e-05 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:382 3.e 3:7303705d:::rbd_data.1152f108f2b3.000000000000001d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.044879+0000", 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0297885569999998, 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018854309999999999, 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:27.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 382 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044879+0000", 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044879+0000", 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044881+0000", 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044890+0000", 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7679999999999997e-06 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044907+0000", 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7362000000000001e-05 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044917+0000", 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1300000000000007e-06 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.044974+0000", 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7621000000000003e-05 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045059+0000", 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4876000000000005e-05 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:27.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045166+0000", 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106847 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045844+0000", 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067806300000000002 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046657+0000", 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081334799999999998 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046717+0000", 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.948e-05 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046765+0000", 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7865999999999999e-05 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:380 3.1 e23/19 3:85745777:::rbd_data.1152f108f2b3.0000000000000013:head v 23'43, mlcod=23'41)", 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.045171+0000", 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0294963369999994, 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010350310000000001, 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045171+0000", 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045171+0000", 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045172+0000", 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6799999999999999e-07 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045181+0000", 2024-09-17T23:29:27.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.687e-06 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045192+0000", 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0696000000000001e-05 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045199+0000", 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045297+0000", 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8596000000000003e-05 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.045319+0000", 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1273000000000001e-05 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046149+0000", 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083028699999999997 2024-09-17T23:29:27.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.046206+0000", 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7546000000000001e-05 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:411 3.1 e23/19 3:86f27224:::rbd_header.1152f108f2b3:head v 23'46, mlcod=23'44)", 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.069987+0000", 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0046810429999997, 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069133200000000001, 2024-09-17T23:29:27.175 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.069987+0000", 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.069987+0000", 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.069987+0000", 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5600000000000001e-07 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.069992+0000", 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.070000+0000", 2024-09-17T23:29:27.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3790000000000006e-06 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.070004+0000", 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.070051+0000", 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6748e-05 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.070074+0000", 2024-09-17T23:29:27.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2551000000000001e-05 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.070639+0000", 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056563000000000002 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.070678+0000", 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8714000000000002e-05 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:427 3.3 e23/19 3:c1f6e925:::rbd_data.1152f108f2b3.0000000000000001:head v 23'1, mlcod=0'0)", 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.077297+0000", 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9973702710000003, 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00088942600000000002, 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077297+0000", 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077297+0000", 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077298+0000", 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1999999999999999e-07 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077344+0000", 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5926e-05 2024-09-17T23:29:27.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077355+0000", 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0471e-05 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077359+0000", 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6840000000000004e-06 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077411+0000", 2024-09-17T23:29:27.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2175000000000001e-05 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.077430+0000", 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8079000000000001e-05 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.078156+0000", 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072639099999999995 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.078187+0000", 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0979999999999998e-05 2024-09-17T23:29:27.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:449 3.14 e23/19 3:282ecb89:::rbd_data.1152f108f2b3.0000000000000018:head v 23'3, mlcod=23'1)", 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.092915+0000", 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.981752878, 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076676000000000001, 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:27.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.092915+0000", 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.092915+0000", 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.092916+0000", 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1500000000000004e-07 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.092938+0000", 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2722e-05 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.092949+0000", 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0750999999999999e-05 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.092955+0000", 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9220000000000002e-06 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.093007+0000", 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2160999999999997e-05 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.093022+0000", 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4598e-05 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.093649+0000", 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00062774900000000004 2024-09-17T23:29:27.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.093682+0000", 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2141999999999998e-05 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:27.188 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:27.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:27.190 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:27.187+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:27.291 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:27.289+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:27.299 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:27.299 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:27.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.299 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.42272198700000002, 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.422743965, 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:29:27.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40709685699999998, 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40716799999999997, 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:29:27.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40705920099999998, 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.407165679, 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:29:27.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40704177000000002, 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40718230300000002, 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:29:27.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:29:27.317 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40700683900000001, 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40718201100000001, 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:27.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.406975419, 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40718365499999998, 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:29:27.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40695879299999999, 2024-09-17T23:29:27.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40720076500000002, 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:27.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:29:27.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40693709900000002, 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40721411000000002, 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.331 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.334 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3.c7944cbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.40692538099999997, 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.40723610100000002, 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.335 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:29:27.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:27.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 9 2024-09-17T23:29:27.339 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:27.340 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:27.392 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:27.390+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:27.447 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:27.447 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:27.456 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:27.493 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:27.492+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:27.594 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:27.592+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1521 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 21364736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.605 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971044+0000", 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6313472280000001, 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011790156, 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1521 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971044+0000", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971044+0000", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971044+0000", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.39e-07 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971046+0000", 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9530000000000002e-06 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971052+0000", 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1469999999999998e-06 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971057+0000", 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0139999999999998e-06 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.982741+0000", 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011683140999999999 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.982776+0000", 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5209999999999997e-05 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.982834+0000", 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8251999999999997e-05 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.609 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1522 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 21889024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971065+0000", 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.631326338, 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011974189, 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1522 2024-09-17T23:29:27.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971065+0000", 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971065+0000", 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971065+0000", 2024-09-17T23:29:27.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.39e-07 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971072+0000", 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2709999999999998e-06 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971078+0000", 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1700000000000002e-06 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971082+0000", 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5399999999999997e-06 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.982866+0000", 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011784204 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.982942+0000", 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5862e-05 2024-09-17T23:29:27.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983039+0000", 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6702999999999999e-05 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1523 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 22413312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971089+0000", 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6313020250000001, 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012048991, 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1523 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971089+0000", 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971089+0000", 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971090+0000", 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5499999999999998e-07 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971092+0000", 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2639999999999998e-06 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971097+0000", 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6470000000000003e-06 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971102+0000", 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3989999999999997e-06 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983051+0000", 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011949194999999999 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983087+0000", 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6229000000000002e-05 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983138+0000", 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0801999999999999e-05 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1524 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 22937600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971109+0000", 2024-09-17T23:29:27.618 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6312823969999997, 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012234945000000001, 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1524 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971109+0000", 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971109+0000", 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971109+0000", 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7300000000000002e-07 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971111+0000", 2024-09-17T23:29:27.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0899999999999999e-06 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971119+0000", 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5739999999999998e-06 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971124+0000", 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9640000000000002e-06 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983171+0000", 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012047288999999999 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983247+0000", 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6100000000000007e-05 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983344+0000", 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6555e-05 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.622 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1525 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 23461888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971128+0000", 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6312629770000004, 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012314138000000001, 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1525 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.623 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971128+0000", 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971128+0000", 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971128+0000", 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.53e-07 2024-09-17T23:29:27.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971130+0000", 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.254e-06 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971133+0000", 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3629999999999998e-06 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971136+0000", 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.593e-06 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983355+0000", 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012219752 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983389+0000", 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4044000000000001e-05 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983442+0000", 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2879e-05 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1526 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 23986176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971139+0000", 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6312523560000001, 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012506613, 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.627 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1526 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971139+0000", 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971139+0000", 2024-09-17T23:29:27.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971139+0000", 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.16e-07 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971140+0000", 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0449999999999999e-06 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971143+0000", 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3550000000000001e-06 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971146+0000", 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.503e-06 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983475+0000", 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012329524999999999 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983548+0000", 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2780000000000005e-05 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983645+0000", 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7188999999999994e-05 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1527 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 24510464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971149+0000", 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6312424070000002, 2024-09-17T23:29:27.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012596092999999999, 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1527 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971149+0000", 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971149+0000", 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971149+0000", 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2599999999999998e-07 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971150+0000", 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5000000000000001e-07 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971154+0000", 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4730000000000002e-06 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971157+0000", 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5670000000000002e-06 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983657+0000", 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.01250019 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983692+0000", 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4647999999999998e-05 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983745+0000", 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2939000000000003e-05 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.635 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1528 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 25034752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971160+0000", 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.631231047, 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012799715999999999, 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1528 2024-09-17T23:29:27.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971160+0000", 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971160+0000", 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971160+0000", 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.16e-07 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971162+0000", 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5829999999999999e-06 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971166+0000", 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2289999999999997e-06 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971169+0000", 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7070000000000001e-06 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983778+0000", 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.01260909 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983860+0000", 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2113000000000005e-05 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983960+0000", 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9778000000000006e-05 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1529 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 25559040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971172+0000", 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6312192769999996, 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.01288649, 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1529 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971172+0000", 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971172+0000", 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971172+0000", 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.16e-07 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971173+0000", 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6299999999999993e-07 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971176+0000", 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9280000000000002e-06 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971178+0000", 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4880000000000001e-06 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.983972+0000", 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012793977999999999 2024-09-17T23:29:27.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984007+0000", 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4715000000000002e-05 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984058+0000", 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1202000000000002e-05 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1530 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 26083328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.631209278, 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013083955, 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1530 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.645 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2999999999999999e-07 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971183+0000", 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1299999999999998e-07 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971186+0000", 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0290000000000001e-06 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971188+0000", 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4279999999999999e-06 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984090+0000", 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012901151 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984168+0000", 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8838000000000006e-05 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984266+0000", 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7366000000000003e-05 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1531 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 26607616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971213+0000", 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6311776590000004, 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013149779, 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1531 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971213+0000", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971213+0000", 2024-09-17T23:29:27.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971214+0000", 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6300000000000001e-07 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971215+0000", 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.528e-06 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971219+0000", 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.557e-06 2024-09-17T23:29:27.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971221+0000", 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5239999999999999e-06 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984278+0000", 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013056885000000001 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984312+0000", 2024-09-17T23:29:27.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3776999999999999e-05 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984363+0000", 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1144999999999999e-05 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1532 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 27131904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.631166436, 2024-09-17T23:29:27.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013345258, 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1532 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0599999999999999e-07 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971226+0000", 2024-09-17T23:29:27.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0589999999999999e-06 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971229+0000", 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0589999999999998e-06 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971231+0000", 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.446e-06 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984396+0000", 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013164110999999999 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984474+0000", 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8658000000000005e-05 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984570+0000", 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5718999999999996e-05 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.657 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1533 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 27656192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971234+0000", 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6311568100000002, 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013433272, 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1533 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971234+0000", 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971234+0000", 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971235+0000", 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2300000000000002e-07 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971236+0000", 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971239+0000", 2024-09-17T23:29:27.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9979999999999999e-06 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971241+0000", 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984582+0000", 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013340604000000001 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984617+0000", 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5660000000000001e-05 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984668+0000", 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0266999999999998e-05 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.662 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1534 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 28180480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6311472670000002, 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.01363252, 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1534 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:27.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1e-07 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971249+0000", 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8239999999999999e-06 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971255+0000", 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971259+0000", 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3900000000000003e-06 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984701+0000", 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0134419 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984782+0000", 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0703000000000003e-05 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984876+0000", 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4767999999999998e-05 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1535 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 28704768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.667 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971265+0000", 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6311256790000002, 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013710455999999999, 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1535 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971265+0000", 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971265+0000", 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971266+0000", 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.13e-07 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971268+0000", 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.933e-06 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971274+0000", 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2230000000000004e-06 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971279+0000", 2024-09-17T23:29:27.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1200000000000001e-06 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984889+0000", 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013609903 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984926+0000", 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6628000000000002e-05 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984976+0000", 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0136000000000002e-05 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1536 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 29229056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6311054890000003, 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013896568, 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1536 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.89e-07 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971288+0000", 2024-09-17T23:29:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.84e-06 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971294+0000", 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.489e-06 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971299+0000", 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9200000000000003e-06 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985010+0000", 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013711008 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985087+0000", 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6694999999999997e-05 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985182+0000", 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5227000000000002e-05 2024-09-17T23:29:27.676 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1537 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 29753344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6310814789999997, 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013987474999999999, 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.677 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1537 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5300000000000004e-07 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971312+0000", 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9149999999999999e-06 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971321+0000", 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5380000000000002e-06 2024-09-17T23:29:27.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971334+0000", 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2962000000000001e-05 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985209+0000", 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013875458 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985247+0000", 2024-09-17T23:29:27.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7829e-05 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985297+0000", 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0219999999999997e-05 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1538 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 30277632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6310285239999995, 2024-09-17T23:29:27.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014136910000000001, 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1538 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000004e-07 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971366+0000", 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5440000000000001e-06 2024-09-17T23:29:27.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971374+0000", 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0190000000000006e-06 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971380+0000", 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8209999999999998e-06 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985329+0000", 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013949088 2024-09-17T23:29:27.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985405+0000", 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5956000000000003e-05 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985500+0000", 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4812e-05 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.685 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1539 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 30801920~64000] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.631003508, 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014374316, 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1539 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8999999999999997e-07 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971396+0000", 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7740000000000004e-06 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971403+0000", 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2459999999999997e-06 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971409+0000", 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4779999999999999e-06 2024-09-17T23:29:27.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985512+0000", 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014103861000000001 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985546+0000", 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3210000000000002e-05 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985762+0000", 2024-09-17T23:29:27.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000216257 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:157 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8256000~55296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.810463+0000", 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.79192799199999997, 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.015970193000000001, 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:27.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 157 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.810463+0000", 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.810463+0000", 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:27.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.810464+0000", 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2699999999999998e-07 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818308+0000", 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078437009999999998 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818322+0000", 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4761e-05 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818329+0000", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6109999999999999e-06 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818400+0000", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1370000000000003e-05 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818497+0000", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6812999999999996e-05 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:27.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818726+0000", 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000228748 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.823365+0000", 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046387970000000001 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.826361+0000", 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00299606 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:27.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.826400+0000", 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8729000000000001e-05 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.826433+0000", 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3776000000000003e-05 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:27.695 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:27.697 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:27.697 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:27.694+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:27.797 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:27.796+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:27.844 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:27.844 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:27.844 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:27.844 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:27.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:29:27.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:27.898 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:27.896+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:27.910 INFO:teuthology.orchestra.run.smithi139.stdout:600 write 0xa9d8600 thru 0xa9d9f0c (0x190d bytes) 2024-09-17T23:29:27.922 INFO:teuthology.orchestra.run.smithi139.stdout:603 trunc from 0xa9da000 to 0x4e54600 2024-09-17T23:29:27.999 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:27.997+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:28.010 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:28.010 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:28.010 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:28.010 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:28.010 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:28.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:28.041 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:28.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:28.074 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:28.090 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:28.100 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:28.098+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:28.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:28.124 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:28.124 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:28.126 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.126 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:28.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:28.201 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:28.199+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:28.282 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.282 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:28.291 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:28.335 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:28.333+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:28.398 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.398 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:28.402 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:28.400+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:28.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:28.503 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:28.502+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:28.554 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.554 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:28.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:28.608 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:28.606+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:28.674 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.675 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:28.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:28.705 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:28.703+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:28.807 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:28.805+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:28.831 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.831 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:28.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:28.907 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:28.905+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:28.967 INFO:teuthology.orchestra.run.smithi139.stdout:604 write 0x7ebd600 thru 0x7ecb2f4 (0xdcf5 bytes) 2024-09-17T23:29:28.977 INFO:teuthology.orchestra.run.smithi139.stdout:605 trunc from 0x7ecb400 to 0xddc5600 2024-09-17T23:29:28.994 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:28.994 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.001 INFO:teuthology.orchestra.run.smithi139.stdout:606 trunc from 0xddc5600 to 0x4af9200 2024-09-17T23:29:29.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:29.018 INFO:teuthology.orchestra.run.smithi139.stdout:607 punch from 0x1aa0800 to 0x1aa9200, (0x8a00 bytes) 2024-09-17T23:29:29.045 INFO:teuthology.orchestra.run.smithi139.stdout:609 write 0xeae6600 thru 0xeaee87e (0x827f bytes) 2024-09-17T23:29:29.045 INFO:teuthology.orchestra.run.smithi139.stdout:610 write 0x2ef800 thru 0x2fd474 (0xdc75 bytes) 2024-09-17T23:29:29.047 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:29.046+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:29.050 INFO:teuthology.orchestra.run.smithi139.stdout:611 read 0x3901200 thru 0x3907b65 (0x6966 bytes) 2024-09-17T23:29:29.050 INFO:teuthology.orchestra.run.smithi139.stdout:612 read 0x304f600 thru 0x3056a61 (0x7462 bytes) 2024-09-17T23:29:29.051 INFO:teuthology.orchestra.run.smithi139.stdout:613 read 0x896b000 thru 0x8973a68 (0x8a69 bytes) 2024-09-17T23:29:29.051 INFO:teuthology.orchestra.run.smithi139.stdout:614 read 0x6352800 thru 0x635b7d1 (0x8fd2 bytes) 2024-09-17T23:29:29.051 INFO:teuthology.orchestra.run.smithi139.stdout:616 punch from 0x928fe00 to 0x9296a00, (0x6c00 bytes) 2024-09-17T23:29:29.052 INFO:teuthology.orchestra.run.smithi139.stdout:618 write 0xea8b000 thru 0xea9a2f3 (0xf2f4 bytes) 2024-09-17T23:29:29.052 INFO:teuthology.orchestra.run.smithi139.stdout:619 read 0x9b2e600 thru 0x9b3ca41 (0xe442 bytes) 2024-09-17T23:29:29.052 INFO:teuthology.orchestra.run.smithi139.stdout:620 write 0xe06d200 thru 0xe072e63 (0x5c64 bytes) 2024-09-17T23:29:29.052 INFO:teuthology.orchestra.run.smithi139.stdout:622 write 0x4ee9800 thru 0x4eef8d6 (0x60d7 bytes) 2024-09-17T23:29:29.053 INFO:teuthology.orchestra.run.smithi139.stdout:623 read 0xad97600 thru 0xada27f0 (0xb1f1 bytes) 2024-09-17T23:29:29.061 INFO:teuthology.orchestra.run.smithi139.stdout:629 trunc from 0xeaeea00 to 0x7aa1000 2024-09-17T23:29:29.081 INFO:teuthology.orchestra.run.smithi139.stdout:631 write 0x26f8800 thru 0x26fb17e (0x297f bytes) 2024-09-17T23:29:29.086 INFO:teuthology.orchestra.run.smithi139.stdout:632 read 0x32a1a00 thru 0x32aec39 (0xd23a bytes) 2024-09-17T23:29:29.087 INFO:teuthology.orchestra.run.smithi139.stdout:634 read 0x3330000 thru 0x3334ac4 (0x4ac5 bytes) 2024-09-17T23:29:29.087 INFO:teuthology.orchestra.run.smithi139.stdout:638 punch from 0x4549800 to 0x454d000, (0x3800 bytes) 2024-09-17T23:29:29.087 INFO:teuthology.orchestra.run.smithi139.stdout:640 write 0x2a72c00 thru 0x2a77c7c (0x507d bytes) 2024-09-17T23:29:29.088 INFO:teuthology.orchestra.run.smithi139.stdout:641 punch from 0x6af5200 to 0x6aff800, (0xa600 bytes) 2024-09-17T23:29:29.095 INFO:teuthology.orchestra.run.smithi139.stdout:642 trunc from 0x7aa1000 to 0x1873000 2024-09-17T23:29:29.109 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:29.107+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:29.112 INFO:teuthology.orchestra.run.smithi139.stdout:643 punch from 0x3bec00 to 0x3c0600, (0x1a00 bytes) 2024-09-17T23:29:29.117 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.117 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.117 INFO:teuthology.orchestra.run.smithi139.stdout:644 read 0x63a600 thru 0x645c40 (0xb641 bytes) 2024-09-17T23:29:29.117 INFO:teuthology.orchestra.run.smithi139.stdout:645 punch from 0x649400 to 0x64f600, (0x6200 bytes) 2024-09-17T23:29:29.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:29.134 INFO:teuthology.orchestra.run.smithi139.stdout:646 write 0x8259400 thru 0x825a5a4 (0x11a5 bytes) 2024-09-17T23:29:29.135 INFO:teuthology.orchestra.run.smithi139.stdout:648 read 0x479fe00 thru 0x47a2ccf (0x2ed0 bytes) 2024-09-17T23:29:29.135 INFO:teuthology.orchestra.run.smithi139.stdout:651 punch from 0x2df6000 to 0x2e04600, (0xe600 bytes) 2024-09-17T23:29:29.136 INFO:teuthology.orchestra.run.smithi139.stdout:652 write 0x5cac00 thru 0x5cec71 (0x4072 bytes) 2024-09-17T23:29:29.136 INFO:teuthology.orchestra.run.smithi139.stdout:655 read 0x309ec00 thru 0x30ab512 (0xc913 bytes) 2024-09-17T23:29:29.137 INFO:teuthology.orchestra.run.smithi139.stdout:660 read 0x6b85800 thru 0x6b927d5 (0xcfd6 bytes) 2024-09-17T23:29:29.137 INFO:teuthology.orchestra.run.smithi139.stdout:667 read 0x4d1c00 thru 0x4d7b8e (0x5f8f bytes) 2024-09-17T23:29:29.137 INFO:teuthology.orchestra.run.smithi139.stdout:671 write 0x3746c00 thru 0x3749e90 (0x3291 bytes) 2024-09-17T23:29:29.156 INFO:teuthology.orchestra.run.smithi139.stdout:672 write 0x9315200 thru 0x931f84d (0xa64e bytes) 2024-09-17T23:29:29.156 INFO:teuthology.orchestra.run.smithi139.stdout:673 punch from 0x1ad4400 to 0x1add200, (0x8e00 bytes) 2024-09-17T23:29:29.157 INFO:teuthology.orchestra.run.smithi139.stdout:675 read 0x3d4da00 thru 0x3d55589 (0x7b8a bytes) 2024-09-17T23:29:29.163 INFO:teuthology.orchestra.run.smithi139.stdout:676 trunc from 0x931fa00 to 0xdcad000 2024-09-17T23:29:29.171 INFO:teuthology.orchestra.run.smithi139.stdout:677 write 0xd0b0200 thru 0xd0b5927 (0x5728 bytes) 2024-09-17T23:29:29.188 INFO:teuthology.orchestra.run.smithi139.stdout:682 trunc from 0xdcad000 to 0xdf3a00 2024-09-17T23:29:29.210 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:29.208+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:29.214 INFO:teuthology.orchestra.run.smithi139.stdout:683 read 0x2ee00 thru 0x34e7e (0x607f bytes) 2024-09-17T23:29:29.221 INFO:teuthology.orchestra.run.smithi139.stdout:684 read 0x84800 thru 0x932c1 (0xeac2 bytes) 2024-09-17T23:29:29.221 INFO:teuthology.orchestra.run.smithi139.stdout:686 read 0xc3f800 thru 0xc49e31 (0xa632 bytes) 2024-09-17T23:29:29.222 INFO:teuthology.orchestra.run.smithi139.stdout:687 read 0x2d9c00 thru 0x2e8a06 (0xee07 bytes) 2024-09-17T23:29:29.248 INFO:teuthology.orchestra.run.smithi139.stdout:691 write 0xed43c00 thru 0xed461c1 (0x25c2 bytes) 2024-09-17T23:29:29.258 INFO:teuthology.orchestra.run.smithi139.stdout:694 trunc from 0xed46200 to 0x6b3a800 2024-09-17T23:29:29.274 INFO:teuthology.orchestra.run.smithi139.stdout:695 read 0x1545c00 thru 0x1549aa8 (0x3ea9 bytes) 2024-09-17T23:29:29.276 INFO:teuthology.orchestra.run.smithi139.stdout:696 read 0x590a400 thru 0x590bcd6 (0x18d7 bytes) 2024-09-17T23:29:29.276 INFO:teuthology.orchestra.run.smithi139.stdout:697 read 0x13bc200 thru 0x13ca1eb (0xdfec bytes) 2024-09-17T23:29:29.276 INFO:teuthology.orchestra.run.smithi139.stdout:699 flatten 2024-09-17T23:29:29.277 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.277 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:29.309 INFO:teuthology.orchestra.run.smithi139.stdout:701 write 0x9db4a00 thru 0x9db6bc4 (0x21c5 bytes) 2024-09-17T23:29:29.320 INFO:teuthology.orchestra.run.smithi139.stdout:703 trunc from 0x9db6c00 to 0x155200 2024-09-17T23:29:29.328 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:29.327+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:29.369 INFO:teuthology.orchestra.run.smithi139.stdout:704 trunc from 0x155200 to 0xea85c00 2024-09-17T23:29:29.377 INFO:teuthology.orchestra.run.smithi139.stdout:705 write 0xd23b000 thru 0xd244059 (0x905a bytes) 2024-09-17T23:29:29.378 INFO:teuthology.orchestra.run.smithi139.stdout:707 read 0x9cd6a00 thru 0x9ce268e (0xbc8f bytes) 2024-09-17T23:29:29.378 INFO:teuthology.orchestra.run.smithi139.stdout:708 write 0xcba5000 thru 0xcbad2d5 (0x82d6 bytes) 2024-09-17T23:29:29.391 INFO:teuthology.orchestra.run.smithi139.stdout:709 trunc from 0xea85c00 to 0x37c8a00 2024-09-17T23:29:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.393 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:29.425 INFO:teuthology.orchestra.run.smithi139.stdout:710 write 0xa06a400 thru 0xa076c03 (0xc804 bytes) 2024-09-17T23:29:29.426 INFO:teuthology.orchestra.run.smithi139.stdout:713 punch from 0x857c400 to 0x857d800, (0x1400 bytes) 2024-09-17T23:29:29.436 INFO:teuthology.orchestra.run.smithi139.stdout:714 trunc from 0xa076e00 to 0x1f5cc00 2024-09-17T23:29:29.446 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:29.444+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:29.452 INFO:teuthology.orchestra.run.smithi139.stdout:715 read 0x3d5600 thru 0x3e176d (0xc16e bytes) 2024-09-17T23:29:29.453 INFO:teuthology.orchestra.run.smithi139.stdout:716 read 0x50dc00 thru 0x516016 (0x8417 bytes) 2024-09-17T23:29:29.457 INFO:teuthology.orchestra.run.smithi139.stdout:717 trunc from 0x1f5cc00 to 0x4f77800 2024-09-17T23:29:29.466 INFO:teuthology.orchestra.run.smithi139.stdout:720 punch from 0x3fdc200 to 0x3fe0600, (0x4400 bytes) 2024-09-17T23:29:29.468 INFO:teuthology.orchestra.run.smithi139.stdout:721 read 0x4923e00 thru 0x493207d (0xe27e bytes) 2024-09-17T23:29:29.468 INFO:teuthology.orchestra.run.smithi139.stdout:723 punch from 0x1a18800 to 0x1a26e00, (0xe600 bytes) 2024-09-17T23:29:29.468 INFO:teuthology.orchestra.run.smithi139.stdout:725 read 0x4acc400 thru 0x4ad3391 (0x6f92 bytes) 2024-09-17T23:29:29.470 INFO:teuthology.orchestra.run.smithi139.stdout:726 read 0x3d34c00 thru 0x3d3cdd5 (0x81d6 bytes) 2024-09-17T23:29:29.471 INFO:teuthology.orchestra.run.smithi139.stdout:728 read 0x2dcc600 thru 0x2dd45fd (0x7ffe bytes) 2024-09-17T23:29:29.474 INFO:teuthology.orchestra.run.smithi139.stdout:730 trunc from 0x4f77800 to 0x2662600 2024-09-17T23:29:29.485 INFO:teuthology.orchestra.run.smithi139.stdout:732 clone 5 order 25 su 524288 sc 10 2024-09-17T23:29:29.513 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:29.512+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:29.553 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.553 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:29.614 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:29.612+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:29.668 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.668 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:29.715 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:29.713+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:29.784 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.784 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:29.836 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:29.834+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:29.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:29:29.901 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:29.902 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:29.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:29.952 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:29.950+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:30.018 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:30.016+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:30.063 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:30.063 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:30.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:30.118 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:30.117+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:30.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:30.219 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:30.217+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:30.221 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:30.221 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:30.230 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:30.232 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T23:29:30.261 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:30.261 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:30.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:30.278 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:30.295 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:30.311 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:30.312 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:30.320 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:30.318+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:30.328 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:30.328 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:30.345 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:30.380 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:30.380 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:30.380 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:30.380 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:30.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:30.421 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:30.419+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:30.522 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:30.520+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:30.536 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:30.536 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:30.536 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:30.536 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:30.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:30.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:30.629 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:30.627+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:30.694 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:97 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:18.754575+0000", 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.936547417, 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0067742569999999997, 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 97 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754575+0000", 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754575+0000", 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754576+0000", 2024-09-17T23:29:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4499999999999996e-07 2024-09-17T23:29:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754589+0000", 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2672e-05 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754603+0000", 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3903000000000001e-05 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.754615+0000", 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1922999999999999e-05 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756053+0000", 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014387270000000001 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756140+0000", 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6750999999999997e-05 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.756431+0000", 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00029059799999999998 2024-09-17T23:29:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.758717+0000", 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022867719999999998 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761261+0000", 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00254404 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761311+0000", 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9347e-05 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:18.761350+0000", 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8779000000000003e-05 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:477 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.578330339000001, 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081640670000000005, 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 477 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112793+0000", 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2e-07 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112796+0000", 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9620000000000002e-06 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112803+0000", 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5989999999999996e-06 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112809+0000", 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.276e-06 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117872+0000", 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050631349999999999 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117960+0000", 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8399000000000005e-05 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118168+0000", 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020731300000000001 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120846+0000", 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026780630000000001 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120889+0000", 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2904000000000001e-05 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120920+0000", 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1075999999999999e-05 2024-09-17T23:29:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120956+0000", 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6820000000000003e-05 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:478 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.578295667000001, 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0088678699999999999, 2024-09-17T23:29:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 478 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2599999999999998e-07 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112831+0000", 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1669999999999999e-06 2024-09-17T23:29:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112839+0000", 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4230000000000005e-06 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112845+0000", 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.02e-06 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119096+0000", 2024-09-17T23:29:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062507580000000004 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119178+0000", 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2100000000000003e-05 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119698+0000", 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00051989400000000002 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121193+0000", 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014956279999999999 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121598+0000", 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00040432800000000002 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121653+0000", 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5541999999999998e-05 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121695+0000", 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1584e-05 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:479 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.578204099000001, 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010595982, 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 479 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3900000000000004e-07 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112923+0000", 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5379999999999998e-06 2024-09-17T23:29:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112931+0000", 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5390000000000002e-06 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112936+0000", 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4809999999999997e-06 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120113+0000", 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0071769390000000002 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120187+0000", 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3967000000000005e-05 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120554+0000", 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036718900000000002 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121301+0000", 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00074710800000000002 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123418+0000", 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021164729999999998 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123475+0000", 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7598e-05 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123515+0000", 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9310999999999997e-05 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:480 3.6 3:641496d6:::rbd_data.1152f108f2b3.0000000000000036:head [read 1069056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.577841866, 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079137459999999993, 2024-09-17T23:29:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 480 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113282+0000", 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3e-07 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113286+0000", 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.228e-06 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113295+0000", 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.38e-06 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113300+0000", 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0010000000000001e-06 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120998+0000", 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0076977790000000001 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121079+0000", 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1055000000000002e-05 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121195+0000", 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115573 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1504 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 26607616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970081+0000", 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.721041295, 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069127490000000002, 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1504 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970081+0000", 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970081+0000", 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970082+0000", 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3300000000000001e-07 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970083+0000", 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.201e-06 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970086+0000", 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2540000000000001e-06 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970089+0000", 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.976803+0000", 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0067145859999999998 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.976896+0000", 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2561999999999999e-05 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.976994+0000", 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8445999999999999e-05 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1505 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 27131904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970092+0000", 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7210309920000002, 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0071205890000000001, 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1505 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970092+0000", 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970092+0000", 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970092+0000", 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2700000000000001e-07 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970093+0000", 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7399999999999991e-07 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970099+0000", 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1059999999999996e-06 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970103+0000", 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3479999999999997e-06 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977027+0000", 2024-09-17T23:29:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069234639999999998 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977103+0000", 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6563999999999994e-05 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977212+0000", 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108906 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1506 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 27656192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970111+0000", 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7210116050000002, 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007311672, 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1506 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970111+0000", 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970111+0000", 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970112+0000", 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2300000000000002e-07 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970114+0000", 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1409999999999999e-06 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970120+0000", 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6989999999999997e-06 2024-09-17T23:29:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970132+0000", 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1525000000000001e-05 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977243+0000", 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007110908 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977313+0000", 2024-09-17T23:29:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0473000000000001e-05 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977423+0000", 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000109503 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1507 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 28180480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970140+0000", 2024-09-17T23:29:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7209827549999996, 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074394370000000001, 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1507 2024-09-17T23:29:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970140+0000", 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970140+0000", 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970140+0000", 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2800000000000002e-07 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970147+0000", 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6950000000000003e-06 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970155+0000", 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6199999999999999e-06 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970160+0000", 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4870000000000002e-06 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977460+0000", 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0072996249999999997 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977518+0000", 2024-09-17T23:29:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8383e-05 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977579+0000", 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1198999999999995e-05 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1508 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 28704768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970168+0000", 2024-09-17T23:29:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.720954667, 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007559128, 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1508 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970168+0000", 2024-09-17T23:29:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970168+0000", 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970169+0000", 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.63e-07 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970171+0000", 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.119e-06 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970178+0000", 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0820000000000001e-06 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970191+0000", 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3671e-05 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977617+0000", 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074253929999999998 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977657+0000", 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0481000000000001e-05 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977727+0000", 2024-09-17T23:29:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9919000000000006e-05 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1509 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 29229056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970199+0000", 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7209236480000003, 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077732410000000002, 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1509 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970199+0000", 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970199+0000", 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970200+0000", 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7599999999999997e-07 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970207+0000", 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2590000000000003e-06 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970214+0000", 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5469999999999999e-06 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970220+0000", 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4820000000000002e-06 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977770+0000", 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0075506410000000003 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977861+0000", 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0414999999999994e-05 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977972+0000", 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111421 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.751 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1510 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 29753344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7208949750000002, 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079238969999999992, 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1510 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:30.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0100000000000005e-07 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970230+0000", 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0190000000000001e-06 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970238+0000", 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0979999999999994e-06 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970243+0000", 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.955e-06 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978015+0000", 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077719240000000004 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978065+0000", 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9289999999999997e-05 2024-09-17T23:29:30.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978152+0000", 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7109999999999998e-05 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1511 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 30277632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970250+0000", 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7208724960000001, 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081269640000000004, 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1511 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970250+0000", 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970250+0000", 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970251+0000", 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3000000000000001e-07 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970253+0000", 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1289999999999999e-06 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970260+0000", 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9419999999999998e-06 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970265+0000", 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1880000000000001e-06 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978197+0000", 2024-09-17T23:29:30.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079315819999999995 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978278+0000", 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1754000000000004e-05 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978377+0000", 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8838999999999993e-05 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1512 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 30801920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970272+0000", 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7208506420000003, 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0082278969999999996, 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1512 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.761 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970272+0000", 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970272+0000", 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970273+0000", 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.46e-07 2024-09-17T23:29:30.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970274+0000", 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.933e-06 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970281+0000", 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8290000000000002e-06 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970286+0000", 2024-09-17T23:29:30.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9679999999999997e-06 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978401+0000", 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081144430000000007 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978444+0000", 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3003000000000001e-05 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978500+0000", 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6274999999999999e-05 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:801 3.7 3:e3b303a3:::rbd_data.1152846a29a0.0000000000000009:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954187+0000", 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.7369353150000002, 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0080268519999999993, 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 801 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954187+0000", 2024-09-17T23:29:30.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954187+0000", 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954188+0000", 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8400000000000003e-07 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954192+0000", 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6969999999999999e-06 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954202+0000", 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9520000000000006e-06 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954208+0000", 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2960000000000004e-06 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954268+0000", 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9689999999999999e-05 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954381+0000", 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011291899999999999 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962214+0000", 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078335139999999998 2024-09-17T23:29:30.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:815 3.19 3:9aba582b:::rbd_data.1152846a29a0.0000000000000011:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954386+0000", 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.7367366520000003, 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079433809999999994, 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.770 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 815 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954386+0000", 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954386+0000", 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954387+0000", 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7599999999999996e-07 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954391+0000", 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1320000000000004e-06 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954403+0000", 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2047e-05 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954411+0000", 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4730000000000003e-06 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954490+0000", 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9579000000000005e-05 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954565+0000", 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4880000000000001e-05 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.955791+0000", 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012257220000000001 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.959032+0000", 2024-09-17T23:29:30.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0032409140000000001 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962228+0000", 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031967789999999999 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962284+0000", 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5472e-05 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962329+0000", 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5507000000000003e-05 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:816 3.1 3:85317552:::rbd_data.1152846a29a0.0000000000000012:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954432+0000", 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.7366907779999998, 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077914200000000003, 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 816 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954432+0000", 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954432+0000", 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954433+0000", 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8100000000000003e-07 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954435+0000", 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8990000000000001e-06 2024-09-17T23:29:30.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954444+0000", 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3799999999999994e-06 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954450+0000", 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9580000000000004e-06 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954738+0000", 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000287841 2024-09-17T23:29:30.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954807+0000", 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.97e-05 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.955737+0000", 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000929446 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.958643+0000", 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029065810000000001 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962117+0000", 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0034735180000000001 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962190+0000", 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2727999999999999e-05 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962223+0000", 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3788000000000002e-05 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:819 3.7 3:e6fade6f:::rbd_data.1152846a29a0.0000000000000014:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954498+0000", 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.736624495, 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078496599999999996, 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 819 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954498+0000", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954498+0000", 2024-09-17T23:29:30.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954499+0000", 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954502+0000", 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3239999999999999e-06 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954511+0000", 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1870000000000003e-06 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954518+0000", 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0729999999999999e-06 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954682+0000", 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00016343600000000001 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954738+0000", 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000003e-05 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954869+0000", 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000130402 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.956765+0000", 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018963750000000001 2024-09-17T23:29:30.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962255+0000", 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054904869999999996 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962310+0000", 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4159000000000001e-05 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962348+0000", 2024-09-17T23:29:30.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8362000000000003e-05 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:325 3.19 3:9df28ccf:::rbd_header.11521d8bcd2d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.875332+0000", 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.8157905259999998, 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010234141, 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.788 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 325 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875332+0000", 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875332+0000", 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875333+0000", 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0299999999999998e-07 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875337+0000", 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2459999999999997e-06 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875347+0000", 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0043e-05 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875353+0000", 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3859999999999998e-06 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875392+0000", 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8841000000000003e-05 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875396+0000", 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.07e-06 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.878416+0000", 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0030207369999999999 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.878498+0000", 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1500000000000002e-05 2024-09-17T23:29:30.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.878686+0000", 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018777900000000001 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.880006+0000", 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013204429999999999 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.885504+0000", 2024-09-17T23:29:30.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054983760000000001 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.885540+0000", 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5410999999999997e-05 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.885566+0000", 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6506e-05 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:438 3.8 3:123517d2:::rbd_data.11521d8bcd2d.0000000000000001:head [stat,truncate 15025664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:27.933392+0000", 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.7577307439999998, 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0083818829999999997, 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 438 2024-09-17T23:29:30.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933392+0000", 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933392+0000", 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:30.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933393+0000", 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.034e-06 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933399+0000", 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6509999999999998e-06 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933411+0000", 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2717e-05 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933420+0000", 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.32e-06 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933478+0000", 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8131000000000003e-05 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for degraded object", 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933769+0000", 2024-09-17T23:29:30.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00029152699999999998 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.936587+0000", 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028173780000000002 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.936675+0000", 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8473000000000005e-05 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:30.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.936841+0000", 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00016582200000000001 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.937289+0000", 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000448307 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.941690+0000", 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.004400798 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.941723+0000", 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2409000000000001e-05 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.941774+0000", 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1316000000000001e-05 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:30.801 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:30.803 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:30.803 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:30.801+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:30.904 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:30.902+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:30.911 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:29:30.932 INFO:teuthology.orchestra.run.smithi139.stdout:733 write 0x40bf200 thru 0x40c278e (0x358f bytes) 2024-09-17T23:29:30.934 INFO:teuthology.orchestra.run.smithi139.stdout:735 read 0x3667000 thru 0x3674a7e (0xda7f bytes) 2024-09-17T23:29:30.934 INFO:teuthology.orchestra.run.smithi139.stdout:736 punch from 0x3a5f400 to 0x3a66600, (0x7200 bytes) 2024-09-17T23:29:30.935 INFO:teuthology.orchestra.run.smithi139.stdout:737 read 0xedee00 thru 0xee94c5 (0xa6c6 bytes) 2024-09-17T23:29:30.936 INFO:teuthology.orchestra.run.smithi139.stdout:738 read 0x3e04e00 thru 0x3e10a1a (0xbc1b bytes) 2024-09-17T23:29:30.936 INFO:teuthology.orchestra.run.smithi139.stdout:739 read 0x1aaf000 thru 0x1abdc4a (0xec4b bytes) 2024-09-17T23:29:30.937 INFO:teuthology.orchestra.run.smithi139.stdout:740 punch from 0x1266000 to 0x126f400, (0x9400 bytes) 2024-09-17T23:29:30.953 INFO:teuthology.orchestra.run.smithi139.stdout:743 write 0x4ee9c00 thru 0x4ef50a5 (0xb4a6 bytes) 2024-09-17T23:29:30.955 INFO:teuthology.orchestra.run.smithi139.stdout:744 punch from 0x25c8800 to 0x25cc400, (0x3c00 bytes) 2024-09-17T23:29:30.961 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:30.961 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:30.961 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:30.961 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:30.970 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:30.976 INFO:teuthology.orchestra.run.smithi139.stdout:745 write 0xc228c00 thru 0xc22fcce (0x70cf bytes) 2024-09-17T23:29:30.977 INFO:teuthology.orchestra.run.smithi139.stdout:746 write 0x3a72c00 thru 0x3a7f99c (0xcd9d bytes) 2024-09-17T23:29:30.977 INFO:teuthology.orchestra.run.smithi139.stdout:748 read 0xb77e00 thru 0xb7d42d (0x562e bytes) 2024-09-17T23:29:30.978 INFO:teuthology.orchestra.run.smithi139.stdout:749 read 0x815d800 thru 0x816cad8 (0xf2d9 bytes) 2024-09-17T23:29:30.978 INFO:teuthology.orchestra.run.smithi139.stdout:753 write 0xab84a00 thru 0xab949bd (0xffbe bytes) 2024-09-17T23:29:30.978 INFO:teuthology.orchestra.run.smithi139.stdout:755 read 0x72a6800 thru 0x72aa800 (0x4001 bytes) 2024-09-17T23:29:30.978 INFO:teuthology.orchestra.run.smithi139.stdout:756 clone 6 order 24 su 524288 sc 5 2024-09-17T23:29:31.005 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:31.003+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:31.106 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:31.104+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:31.118 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:31.118 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:31.118 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:31.118 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:31.118 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:31.127 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:31.207 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:31.206+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:31.275 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1793 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013893+0000", 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2588636320000006, 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017114821999999998, 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1793 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013893+0000", 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013893+0000", 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013893+0000", 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013900+0000", 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4139999999999999e-06 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013904+0000", 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2139999999999998e-06 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013906+0000", 2024-09-17T23:29:31.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5629999999999999e-06 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.030833+0000", 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.016926961000000001 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.030909+0000", 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6188999999999998e-05 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031008+0000", 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8262000000000002e-05 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1794 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013910+0000", 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2588464699999999, 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017235446000000001, 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1794 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.282 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013910+0000", 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013910+0000", 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013910+0000", 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2399999999999999e-07 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013911+0000", 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013914+0000", 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.106e-06 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013917+0000", 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031037+0000", 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017119872000000001 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031087+0000", 2024-09-17T23:29:31.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0336000000000001e-05 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031145+0000", 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8359000000000001e-05 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1795 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.286 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2588364940000005, 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017441358000000001, 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1795 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.04e-07 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013921+0000", 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9699999999999994e-07 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013924+0000", 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9409999999999999e-06 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013927+0000", 2024-09-17T23:29:31.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.435e-06 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031185+0000", 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017258909999999999 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031263+0000", 2024-09-17T23:29:31.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7015999999999996e-05 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031361+0000", 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8855000000000001e-05 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1796 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:31.291 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2588260610000006, 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017564354000000001, 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1796 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013931+0000", 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-07 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013935+0000", 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5859999999999998e-06 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013939+0000", 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.473e-06 2024-09-17T23:29:31.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013941+0000", 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.492e-06 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031391+0000", 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017449416999999998 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031438+0000", 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6885999999999998e-05 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031495+0000", 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7281999999999998e-05 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.296 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1797 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 9830400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2588117299999997, 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017763582999999999, 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1797 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.297 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2600000000000001e-07 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013946+0000", 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.071e-06 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013949+0000", 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1020000000000001e-06 2024-09-17T23:29:31.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013952+0000", 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1039999999999998e-06 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031533+0000", 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017580312000000001 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031612+0000", 2024-09-17T23:29:31.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9287000000000002e-05 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031708+0000", 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6481e-05 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1798 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.301 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2588010260000004, 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017889677, 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1798 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013956+0000", 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3900000000000001e-07 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013957+0000", 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.093e-06 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013960+0000", 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0989999999999999e-06 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013969+0000", 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.67e-06 2024-09-17T23:29:31.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031737+0000", 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017768055000000001 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031784+0000", 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7057e-05 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031845+0000", 2024-09-17T23:29:31.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1464000000000006e-05 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1799 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2587726749999995, 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018073763999999999, 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.306 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1799 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0900000000000002e-07 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013987+0000", 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.379e-06 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013994+0000", 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4429999999999997e-06 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014000+0000", 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3469999999999998e-06 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031883+0000", 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017883546 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031962+0000", 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9000999999999998e-05 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032058+0000", 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5538999999999994e-05 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1800 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.310 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2587473310000004, 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018179992999999998, 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1800 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014010+0000", 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8299999999999997e-07 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014012+0000", 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1160000000000002e-06 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014020+0000", 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3340000000000002e-06 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014025+0000", 2024-09-17T23:29:31.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2429999999999996e-06 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032086+0000", 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018060579 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032132+0000", 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6464000000000001e-05 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032189+0000", 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6774000000000002e-05 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1801 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2587229329999996, 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018361467999999999, 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1801 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999999999999999e-07 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014036+0000", 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:29:31.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014044+0000", 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1829999999999994e-06 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014051+0000", 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9260000000000003e-06 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032228+0000", 2024-09-17T23:29:31.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018176906 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032304+0000", 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939000000000006e-05 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032395+0000", 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0845000000000002e-05 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1803 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2586956659999995, 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018464566000000002, 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1803 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.320 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014063+0000", 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0779999999999998e-06 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014078+0000", 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4413000000000001e-05 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014084+0000", 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032429+0000", 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018344972000000001 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032469+0000", 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0179000000000003e-05 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032525+0000", 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6561999999999998e-05 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1805 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2586535990000005, 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018638358000000001, 2024-09-17T23:29:31.324 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1805 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4099999999999999e-07 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014106+0000", 2024-09-17T23:29:31.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6460000000000002e-06 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014116+0000", 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8689999999999999e-06 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014124+0000", 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2139999999999996e-06 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032564+0000", 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018440070999999999 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032639+0000", 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4868000000000002e-05 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032741+0000", 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102149 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1810 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2586222770000006, 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018741180999999999, 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1810 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014135+0000", 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014144+0000", 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0629999999999995e-06 2024-09-17T23:29:31.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014151+0000", 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5059999999999998e-06 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014163+0000", 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1399e-05 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032774+0000", 2024-09-17T23:29:31.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018610924000000001 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032818+0000", 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3927999999999997e-05 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032875+0000", 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7896000000000003e-05 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.333 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1814 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2585848760000005, 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018916773000000001, 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1814 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6300000000000005e-07 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014175+0000", 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4739999999999999e-06 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014182+0000", 2024-09-17T23:29:31.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0990000000000001e-06 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014187+0000", 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2630000000000003e-06 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032915+0000", 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018727573000000001 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032990+0000", 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5766000000000006e-05 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033088+0000", 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7935000000000003e-05 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1822 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.338 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2585604700000008, 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019019635999999999, 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1822 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:31.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6199999999999998e-07 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014199+0000", 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3309999999999998e-06 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014206+0000", 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.148e-06 2024-09-17T23:29:31.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014212+0000", 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9490000000000001e-06 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033118+0000", 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018906164 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033159+0000", 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1230999999999999e-05 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033216+0000", 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6351000000000003e-05 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.343 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1826 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2585298999999992, 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019198232999999999, 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1826 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:31.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014229+0000", 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8419999999999999e-06 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014233+0000", 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1150000000000004e-06 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014242+0000", 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1900000000000001e-06 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033254+0000", 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019012228999999999 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033328+0000", 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3709999999999997e-05 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033425+0000", 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6754000000000002e-05 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1830 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2585096839999999, 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019304154, 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1830 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.15e-07 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014252+0000", 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0939999999999998e-06 2024-09-17T23:29:31.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014257+0000", 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3610000000000003e-06 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014259+0000", 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6350000000000002e-06 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033453+0000", 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019193661000000001 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033495+0000", 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033551+0000", 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168000000000002e-05 2024-09-17T23:29:31.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1834 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2584933399999993, 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019503455999999999, 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.353 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1834 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:31.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4499999999999998e-07 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014265+0000", 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1400000000000001e-06 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014269+0000", 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6149999999999999e-06 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014272+0000", 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5220000000000002e-06 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033591+0000", 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019318887 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033668+0000", 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7348000000000006e-05 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033767+0000", 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8698999999999998e-05 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1837 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2584806470000007, 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019625881000000001, 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1837 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.358 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:31.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.28e-07 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014277+0000", 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.145e-06 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014280+0000", 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1130000000000001e-06 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014283+0000", 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3369999999999998e-06 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033798+0000", 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019515498999999999 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033844+0000", 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5553000000000002e-05 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033902+0000", 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8006e-05 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1839 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2584701369999998, 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019823990999999999, 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1839 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014287+0000", 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014288+0000", 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.133e-06 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014291+0000", 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0479999999999999e-06 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014294+0000", 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9160000000000001e-06 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033941+0000", 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019646961000000001 2024-09-17T23:29:31.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034015+0000", 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4411000000000003e-05 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034110+0000", 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5313e-05 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1841 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.2584594560000006, 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019940504000000001, 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1841 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.367 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:31.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2999999999999999e-07 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014298+0000", 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0249999999999999e-06 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014302+0000", 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014304+0000", 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3209999999999999e-06 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034143+0000", 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019838880999999999 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034183+0000", 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9988999999999999e-05 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:31.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034238+0000", 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4833999999999997e-05 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:31.371 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:31.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:31.373 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:31.371+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:31.473 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:31.471+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:31.492 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:31.493 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:31.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:31.587 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:31.585+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:31.652 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:31.653 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:31.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:31.674 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:31.672+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:31.775 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:31.773+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:31.811 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4236.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.305840017000001, 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046920809999999999, 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4236", 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/2908599310", 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502232+0000", 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502234+0000", 2024-09-17T23:29:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9400000000000001e-06 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502243+0000", 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4290000000000007e-06 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502260+0000", 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6872e-05 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502272+0000", 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1792e-05 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502599+0000", 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032732099999999999 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502674+0000", 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.436e-05 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.502795+0000", 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121549 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.504847+0000", 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020513620000000002 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506847+0000", 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020007689999999999 2024-09-17T23:29:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506887+0000", 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9647000000000002e-05 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:07.506924+0000", 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7039999999999998e-05 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4447.0:175 3.b e23/19 3:d3468d19:::rbd_data.1152f108f2b3.0000000000000009:head v 23'2, mlcod=0'0)", 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950722924000001, 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031410980000000002, 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857349+0000", 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857350+0000", 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857354+0000", 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1520000000000002e-06 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857361+0000", 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8299999999999996e-06 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857367+0000", 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857408+0000", 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1065000000000001e-05 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.857425+0000", 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6977999999999999e-05 2024-09-17T23:29:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.860455+0000", 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030295080000000002 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:18.860490+0000", 2024-09-17T23:29:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5401999999999997e-05 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4447.0:355 3.9 3:91b1f5bf:::rbd_data.1152f108f2b3.0000000000000029:head [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.794590704000001, 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029455660000000002, 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4447", 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 355 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013481+0000", 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013482+0000", 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0529999999999999e-06 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013494+0000", 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1239e-05 2024-09-17T23:29:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013508+0000", 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4321000000000001e-05 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013518+0000", 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8320000000000001e-06 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013564+0000", 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6347999999999997e-05 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.013617+0000", 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2679000000000002e-05 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:19.016427+0000", 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002810094 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4468.0:910 3.f 3:f7e9d27a:::rbd_object_map.11522ab9a303:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.380485974000001, 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003778319, 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4468", 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/4195800266", 2024-09-17T23:29:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 910 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427586+0000", 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427587+0000", 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999998e-07 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427591+0000", 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9160000000000003e-06 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427600+0000", 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5740000000000002e-06 2024-09-17T23:29:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427606+0000", 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9220000000000002e-06 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427654+0000", 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8059999999999997e-05 2024-09-17T23:29:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.427748+0000", 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3789999999999995e-05 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.430511+0000", 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027624400000000001 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431143+0000", 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00063282300000000002 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431309+0000", 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016585999999999999 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431340+0000", 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0731000000000001e-05 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:21.431364+0000", 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.44e-05 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2049 3.a 3:52cea45b:::rbd_data.1152fda325c3.0000000000000008:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.713164913, 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002789297, 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2049 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094907+0000", 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094908+0000", 2024-09-17T23:29:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094911+0000", 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-06 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094919+0000", 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8009999999999993e-06 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094924+0000", 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4469999999999999e-06 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.094967+0000", 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3343000000000002e-05 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.095505+0000", 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053770000000000001 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.095622+0000", 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117087 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.096354+0000", 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073242800000000003 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097606+0000", 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001251432 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097660+0000", 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3693000000000002e-05 2024-09-17T23:29:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.097696+0000", 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6807000000000002e-05 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2378 3.c 3:36bb075d:::rbd_header.1152fda325c3:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6543359730000002, 2024-09-17T23:29:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024627289999999999, 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2378 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153736+0000", 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153737+0000", 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5600000000000005e-07 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153740+0000", 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0319999999999999e-06 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153746+0000", 2024-09-17T23:29:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153750+0000", 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6710000000000001e-06 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153784+0000", 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4799999999999999e-05 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153830+0000", 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5719000000000001e-05 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.153947+0000", 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011649100000000001 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.154698+0000", 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075154900000000001 2024-09-17T23:29:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156144+0000", 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445794 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156173+0000", 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9436000000000001e-05 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.156199+0000", 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-05 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2477 3.17 e26/19 3:eee396ce:::rbd_object_map.1152fda325c3:head v 26'40, mlcod=26'38)", 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6145659489999993, 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034184390000000001, 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193506+0000", 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193514+0000", 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193521+0000", 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1230000000000002e-06 2024-09-17T23:29:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193526+0000", 2024-09-17T23:29:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1780000000000002e-06 2024-09-17T23:29:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193596+0000", 2024-09-17T23:29:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9776000000000004e-05 2024-09-17T23:29:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193618+0000", 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2603e-05 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196879+0000", 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032604230000000001 2024-09-17T23:29:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196924+0000", 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5816000000000003e-05 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2478 3.12 e26/19 3:4952c6c2:::rbd_data.1152fda325c3.0000000000000000:head v 26'9, mlcod=26'7)", 2024-09-17T23:29:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6144499930000009, 2024-09-17T23:29:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032918359999999998, 2024-09-17T23:29:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193622+0000", 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-09-17T23:29:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193652+0000", 2024-09-17T23:29:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9592000000000001e-05 2024-09-17T23:29:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193658+0000", 2024-09-17T23:29:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9669999999999998e-06 2024-09-17T23:29:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193662+0000", 2024-09-17T23:29:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6959999999999998e-06 2024-09-17T23:29:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193711+0000", 2024-09-17T23:29:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9685000000000003e-05 2024-09-17T23:29:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.193724+0000", 2024-09-17T23:29:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2999999999999999e-05 2024-09-17T23:29:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196886+0000", 2024-09-17T23:29:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031617390000000002 2024-09-17T23:29:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.196914+0000", 2024-09-17T23:29:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7724000000000001e-05 2024-09-17T23:29:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2529 3.15 3:af4fb69a:::rbd_data.1152fda325c3.0000000000000001:head [stat,truncate 10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5849117170000007, 2024-09-17T23:29:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054525839999999999, 2024-09-17T23:29:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2529 2024-09-17T23:29:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223160+0000", 2024-09-17T23:29:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223161+0000", 2024-09-17T23:29:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7299999999999996e-07 2024-09-17T23:29:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223164+0000", 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0709999999999999e-06 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223172+0000", 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0450000000000001e-06 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223176+0000", 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2950000000000004e-06 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223233+0000", 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6994999999999999e-05 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223318+0000", 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4400999999999994e-05 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223467+0000", 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000149796 2024-09-17T23:29:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224157+0000", 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000689465 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228516+0000", 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004358682 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228573+0000", 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7596999999999998e-05 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228613+0000", 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9663999999999998e-05 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2531 3.17 e26/19 3:ec02a8b5:::rbd_data.1152fda325c3.0000000000000003:head v 26'43, mlcod=26'41)", 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5846269320000008, 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051349280000000004, 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223445+0000", 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223451+0000", 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7849999999999997e-06 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223458+0000", 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4150000000000004e-06 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223461+0000", 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3390000000000001e-06 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223501+0000", 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0201999999999999e-05 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.223521+0000", 2024-09-17T23:29:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0085000000000002e-05 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228525+0000", 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050040179999999998 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228580+0000", 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4688999999999998e-05 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2532 3.1b e26/19 3:d90b4fac:::rbd_data.1152fda325c3.0000000000000004:head v 26'13, mlcod=26'11)", 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5832869590000005, 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037986270000000002, 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224785+0000", 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224789+0000", 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8909999999999997e-06 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224797+0000", 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224800+0000", 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.63e-06 2024-09-17T23:29:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224844+0000", 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3661000000000001e-05 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.224858+0000", 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3709e-05 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228528+0000", 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003670576 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.228584+0000", 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5461999999999999e-05 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2584 3.f 3:f4c75701:::rbd_data.1152fda325c3.0000000000000006:head [stat,truncate 6815744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5584444160000004, 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041927789999999998, 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2584 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249628+0000", 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:29:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249631+0000", 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9059999999999998e-06 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249644+0000", 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2934e-05 2024-09-17T23:29:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249648+0000", 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249690+0000", 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999998e-05 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249765+0000", 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4518999999999996e-05 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.249913+0000", 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001483 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253747+0000", 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003834306 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253767+0000", 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0047e-05 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253798+0000", 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1090999999999998e-05 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253820+0000", 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1963999999999998e-05 2024-09-17T23:29:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2577 3.12 e26/19 3:4952c6c2:::rbd_data.1152fda325c3.0000000000000000:head v 26'12, mlcod=26'10)", 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.250165+0000", 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5579066220000009, 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035971060000000001, 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250165+0000", 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250165+0000", 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250166+0000", 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2399999999999997e-07 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250180+0000", 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3444999999999999e-05 2024-09-17T23:29:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250190+0000", 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0633e-05 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250197+0000", 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1559999999999998e-06 2024-09-17T23:29:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250244+0000", 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6922000000000002e-05 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250265+0000", 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0797000000000001e-05 2024-09-17T23:29:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253731+0000", 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00346605 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253762+0000", 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1279000000000003e-05 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2580 3.17 e26/19 3:ec02a8b5:::rbd_data.1152fda325c3.0000000000000003:head v 26'45, mlcod=26'43)", 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5578574619999994, 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036269259999999999, 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250215+0000", 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250219+0000", 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6849999999999999e-06 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250225+0000", 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250230+0000", 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250287+0000", 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6539000000000002e-05 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250304+0000", 2024-09-17T23:29:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7439e-05 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253824+0000", 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035195550000000002 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253841+0000", 2024-09-17T23:29:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7793999999999999e-05 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2581 3.1b e26/19 3:d90b4fac:::rbd_data.1152fda325c3.0000000000000004:head v 26'14, mlcod=26'12)", 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5578230699999995, 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035399889999999999, 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250249+0000", 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1799999999999995e-07 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250258+0000", 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0870000000000007e-06 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250267+0000", 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2240000000000001e-06 2024-09-17T23:29:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250272+0000", 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6269999999999997e-06 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250324+0000", 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1838000000000001e-05 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.250342+0000", 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7537000000000002e-05 2024-09-17T23:29:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253749+0000", 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003407546 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.253789+0000", 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9612e-05 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:2690 3.a 3:52cea45b:::rbd_data.1152fda325c3.0000000000000008:head [set-alloc-hint object_size 16777216 write_size 16777216,write 2974720~3584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.296602+0000", 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5114698569999998, 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00246517, 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2690 2024-09-17T23:29:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296602+0000", 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296602+0000", 2024-09-17T23:29:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296603+0000", 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0699999999999996e-07 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296607+0000", 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3780000000000003e-06 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296617+0000", 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3610000000000007e-06 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296621+0000", 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8300000000000003e-06 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296674+0000", 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2225999999999998e-05 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296751+0000", 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7873999999999993e-05 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.296898+0000", 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014618399999999999 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.298802+0000", 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904053 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.298972+0000", 2024-09-17T23:29:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000170318 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.299028+0000", 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6289999999999998e-05 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.299067+0000", 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8949000000000003e-05 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4486.0:2703 3.17 e26/19 3:eee396ce:::rbd_object_map.1152fda325c3:head v 26'54, mlcod=26'52)", 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.322049+0000", 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.4860231640000006, 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003613434, 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322049+0000", 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322049+0000", 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322050+0000", 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5570000000000001e-06 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322061+0000", 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0672999999999999e-05 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322086+0000", 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.478e-05 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322098+0000", 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2186e-05 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322203+0000", 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000105099 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.322237+0000", 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3716e-05 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.325527+0000", 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032903479999999998 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.325662+0000", 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013507499999999999 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4551.0:758 3.e 3:76bd1ce5:::rbd_object_map.1152846a29a0.0000000000000006:head [writefull 0~47] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:25.871468+0000", 2024-09-17T23:29:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9366039580000001, 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042291869999999997, 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4551", 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 758 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871468+0000", 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871468+0000", 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871470+0000", 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3580000000000001e-06 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871486+0000", 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5542000000000001e-05 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871508+0000", 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1999999999999999e-05 2024-09-17T23:29:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871520+0000", 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1858999999999999e-05 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871639+0000", 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119356 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.871644+0000", 2024-09-17T23:29:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.874231+0000", 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025869880000000001 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.874363+0000", 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131667 2024-09-17T23:29:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.874508+0000", 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014457399999999999 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875352+0000", 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084419200000000001 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875597+0000", 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024543600000000003 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875653+0000", 2024-09-17T23:29:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6178000000000003e-05 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.875697+0000", 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3890999999999997e-05 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4551.0:804 3.3 3:c527221d:::rbd_data.1152846a29a0.000000000000000b:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:25.954264+0000", 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8538074819999997, 2024-09-17T23:29:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00297993, 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4551", 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 804 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954264+0000", 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954264+0000", 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954266+0000", 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.124e-06 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954270+0000", 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5630000000000002e-06 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954281+0000", 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1082e-05 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954287+0000", 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.716e-06 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954353+0000", 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5656999999999998e-05 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.954477+0000", 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124468 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.957244+0000", 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027673200000000002 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4551.0:826 3.3 3:c4f02a04:::rbd_data.1152846a29a0.0000000000000021:head [delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:25.955136+0000", 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8529355469999995, 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041672549999999999, 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4551", 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 826 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955136+0000", 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955136+0000", 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955138+0000", 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.156e-06 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955143+0000", 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2050000000000001e-06 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955167+0000", 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.457e-05 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955180+0000", 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2588e-05 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955248+0000", 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7714999999999998e-05 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955321+0000", 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.955489+0000", 2024-09-17T23:29:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167746 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.957310+0000", 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018216910000000001 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.959209+0000", 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001898336 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.959264+0000", 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5766000000000001e-05 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:25.959304+0000", 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9239000000000002e-05 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:31.947 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:31.947 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:31.947 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:31.947 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T23:29:31.948 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.349466 2024-09-17T23:29:31.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.3494659505928882 2024-09-17T23:29:31.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:31.950 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:31.946+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:31.998 INFO:teuthology.orchestra.run.smithi139.stdout:flattening image image_client.0-clone5 2024-09-17T23:29:32.048 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:32.047+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:32.101 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:32.101 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:32.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:32.152 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:32.150+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:32.249 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:32.248+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:32.260 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:32.260 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:32.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:32.353 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:32.351+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:32.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:32.372 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.946874944000001, 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1858 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.405791657, 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010693322999999999, 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1858 2024-09-17T23:29:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.22e-07 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012524+0000", 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1209999999999999e-06 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012527+0000", 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2490000000000001e-06 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012530+0000", 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5579999999999999e-06 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023039+0000", 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010508671000000001 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023116+0000", 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7574e-05 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023216+0000", 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9927999999999996e-05 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1859 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4057452769999994, 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010887203, 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1859 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012570+0000", 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012571+0000", 2024-09-17T23:29:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.75e-06 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012578+0000", 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012581+0000", 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023262+0000", 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010680640999999999 2024-09-17T23:29:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023344+0000", 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2785000000000001e-05 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023456+0000", 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111983 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1861 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4057291579999998, 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011093442, 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1861 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012586+0000", 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012590+0000", 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.724e-06 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012594+0000", 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6320000000000001e-06 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012597+0000", 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6769999999999999e-06 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023495+0000", 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010898309 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023580+0000", 2024-09-17T23:29:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023679+0000", 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9230999999999999e-05 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1862 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4057143829999994, 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011325251, 2024-09-17T23:29:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1862 2024-09-17T23:29:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0600000000000001e-07 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012602+0000", 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2589999999999999e-06 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012605+0000", 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.072e-06 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012607+0000", 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023725+0000", 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01111767 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023810+0000", 2024-09-17T23:29:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840999999999996e-05 2024-09-17T23:29:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023925+0000", 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011567500000000001 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1863 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4057043050000004, 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011535393, 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1863 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-07 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012612+0000", 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1799999999999999e-06 2024-09-17T23:29:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012615+0000", 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012617+0000", 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5000000000000002e-06 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023967+0000", 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011349704 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024045+0000", 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7955999999999997e-05 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024146+0000", 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100691 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1864 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 9830400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056942380000006, 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01177426, 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1864 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1400000000000001e-07 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012622+0000", 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-06 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012625+0000", 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.128e-06 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012627+0000", 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024192+0000", 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011564606999999999 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024282+0000", 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9752000000000004e-05 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024395+0000", 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011288199999999999 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1865 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056841359999996, 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011984761999999999, 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1865 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012631+0000", 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012632+0000", 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.215e-06 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012635+0000", 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1489999999999998e-06 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012637+0000", 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.57e-06 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024435+0000", 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011797368000000001 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024512+0000", 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7081000000000003e-05 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024615+0000", 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103176 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1866 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056740869999995, 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012220981000000001, 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1866 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012641+0000", 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012642+0000", 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1230000000000001e-06 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012645+0000", 2024-09-17T23:29:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.18e-06 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012647+0000", 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4329999999999998e-06 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024660+0000", 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012012372 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024747+0000", 2024-09-17T23:29:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7374999999999996e-05 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024861+0000", 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011429000000000001 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1867 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056640250000001, 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012428603999999999, 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1867 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012651+0000", 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012652+0000", 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1969999999999999e-06 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012655+0000", 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1530000000000001e-06 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012658+0000", 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.543e-06 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024903+0000", 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012244994 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024977+0000", 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3951999999999999e-05 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025079+0000", 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102546 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1868 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056521489999998, 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012658264000000001, 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1868 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012663+0000", 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012664+0000", 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012667+0000", 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1839999999999999e-06 2024-09-17T23:29:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012669+0000", 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4990000000000001e-06 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025123+0000", 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012453138000000001 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025207+0000", 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4153000000000006e-05 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025321+0000", 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113874 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1869 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056419340000001, 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012863466, 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1869 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012674+0000", 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.111e-06 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012677+0000", 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1190000000000001e-06 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012680+0000", 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5809999999999999e-06 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025360+0000", 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012680593 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025436+0000", 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5768999999999999e-05 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025536+0000", 2024-09-17T23:29:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100068 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1870 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056318450000003, 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013108685, 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1870 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012684+0000", 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0890000000000001e-06 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012687+0000", 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.146e-06 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012690+0000", 2024-09-17T23:29:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5519999999999999e-06 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025589+0000", 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012899713 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025676+0000", 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6457000000000002e-05 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025791+0000", 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115516 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1871 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056217590000006, 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013256860000000001, 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1871 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012695+0000", 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.855e-06 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012698+0000", 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2600000000000001e-06 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012701+0000", 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025836+0000", 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013135338999999999 2024-09-17T23:29:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025884+0000", 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7846999999999998e-05 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025950+0000", 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5791999999999996e-05 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1872 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056106340000003, 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013498906, 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1872 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012705+0000", 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2619999999999999e-06 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012709+0000", 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1990000000000002e-06 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012711+0000", 2024-09-17T23:29:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4310000000000001e-06 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025996+0000", 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013284921 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026084+0000", 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7754e-05 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026203+0000", 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119133 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1873 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4056004360000003, 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013625369999999999, 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1873 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-07 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012723+0000", 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6079999999999996e-06 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012727+0000", 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0319999999999997e-06 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012730+0000", 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7590000000000002e-06 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026236+0000", 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013505980000000001 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026279+0000", 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3838000000000003e-05 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026339+0000", 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9953e-05 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1874 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 15073280~31744] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4055805110000001, 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013877722, 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1874 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4699999999999998e-07 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012736+0000", 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3740000000000001e-06 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012739+0000", 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3019999999999999e-06 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012741+0000", 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5739999999999998e-06 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026384+0000", 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013642965 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026465+0000", 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:29:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026612+0000", 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014634700000000001 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1875 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4055698939999992, 2024-09-17T23:29:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013997559, 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1875 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-07 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012746+0000", 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2729999999999999e-06 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012749+0000", 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012752+0000", 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.672e-06 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026646+0000", 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013894098000000001 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026687+0000", 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0540000000000001e-05 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026742+0000", 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5585999999999999e-05 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1876 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4055594540000005, 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014222617999999999, 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1876 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3300000000000001e-07 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012757+0000", 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.353e-06 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012760+0000", 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1930000000000002e-06 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012762+0000", 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026784+0000", 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014021909000000001 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026867+0000", 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3014000000000002e-05 2024-09-17T23:29:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026978+0000", 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110282 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1877 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4055226360000006, 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014306071, 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1877 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3799999999999998e-07 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012794+0000", 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4169999999999999e-06 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012800+0000", 2024-09-17T23:29:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.896e-06 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012803+0000", 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2760000000000001e-06 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027001+0000", 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014198555999999999 2024-09-17T23:29:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027040+0000", 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8479000000000002e-05 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027098+0000", 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8109000000000002e-05 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:32.519 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:32.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:32.521 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:32.521 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:32.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:32.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:32.522 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:32.523 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:32.518+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:32.527 INFO:teuthology.orchestra.run.smithi139.stdout:757 read 0xbe8ec00 thru 0xbe90706 (0x1b07 bytes) 2024-09-17T23:29:32.530 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:32.530 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:32.544 INFO:teuthology.orchestra.run.smithi139.stdout:759 write 0xe661400 thru 0xe66c8d6 (0xb4d7 bytes) 2024-09-17T23:29:32.544 INFO:teuthology.orchestra.run.smithi139.stdout:761 write 0x7c1600 thru 0x7cdb3d (0xc53e bytes) 2024-09-17T23:29:32.545 INFO:teuthology.orchestra.run.smithi139.stdout:765 write 0xb68d000 thru 0xb69a4f1 (0xd4f2 bytes) 2024-09-17T23:29:32.545 INFO:teuthology.orchestra.run.smithi139.stdout:767 read 0x75fac00 thru 0x76081a2 (0xd5a3 bytes) 2024-09-17T23:29:32.546 INFO:teuthology.orchestra.run.smithi139.stdout:769 punch from 0x5326800 to 0x5331a00, (0xb200 bytes) 2024-09-17T23:29:32.546 INFO:teuthology.orchestra.run.smithi139.stdout:772 write 0x6797200 thru 0x6798d5d (0x1b5e bytes) 2024-09-17T23:29:32.547 INFO:teuthology.orchestra.run.smithi139.stdout:773 clone 7 order 20 su 65536 sc 7 2024-09-17T23:29:32.549 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:32.550 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:32.568 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:32.568 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:32.620 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:32.618+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:32.634 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:32.634 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:32.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:32.721 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:32.719+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:32.749 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:32.750 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:32.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:32.822 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:32.820+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:32.865 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:32.865 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:32.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:32.922 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 0.349466 (8229022) 2024-09-17T23:29:32.923 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:32.921+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:32.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:29:32.987 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:32.987 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:32.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:33.025 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:33.022+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:33.125 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:33.123+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:33.143 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.143 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.152 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:33.226 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:33.224+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:33.258 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.258 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:33.327 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:33.325+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:33.374 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.375 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:33.467 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:33.465+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:33.529 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:33.528+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:33.530 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.530 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.538 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:33.631 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:33.629+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:33.686 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.686 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:33.732 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:33.730+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:33.802 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.802 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:33.855 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:33.853+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:33.919 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:33.919 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:33.921 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:29:33.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:33.969 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:33.967+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:34.001 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone6 from 0xe66ca00 (overlap 0xc22fe00) to 0x0 2024-09-17T23:29:34.033 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:34.031+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:34.080 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:34.080 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:34.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:34.134 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:34.132+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:34.198 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:34.199 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:34.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:34.249 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:34.247+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:34.335 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:34.333+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:34.347 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T23:29:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:34.357 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:34.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:34.436 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:34.434+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:34.523 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:34.523 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:34.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:34.537 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:34.535+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:34.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:35.027 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:35.027 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:35.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:775 read 0x6b58200 thru 0x6b668e9 (0xe6ea bytes) 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:776 punch from 0xb070800 to 0xb073200, (0x2a00 bytes) 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:777 trunc from 0xe66ca00 to 0xdc2b600 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:779 punch from 0x72bf400 to 0x72cde00, (0xea00 bytes) 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:782 trunc from 0xdc2b600 to 0xd09c600 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:784 trunc from 0xd09c600 to 0x8ddbc00 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:786 trunc from 0x8ddbc00 to 0x99bb200 2024-09-17T23:29:35.028 INFO:teuthology.orchestra.run.smithi139.stdout:788 trunc from 0x99bb200 to 0x868f000 2024-09-17T23:29:35.029 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:35.024+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:35.056 INFO:teuthology.orchestra.run.smithi139.stdout:790 read 0x7755600 thru 0x775debe (0x88bf bytes) 2024-09-17T23:29:35.058 INFO:teuthology.orchestra.run.smithi139.stdout:792 read 0x2049c00 thru 0x204cb00 (0x2f01 bytes) 2024-09-17T23:29:35.058 INFO:teuthology.orchestra.run.smithi139.stdout:793 punch from 0x6c56800 to 0x6c5f000, (0x8800 bytes) 2024-09-17T23:29:35.070 INFO:teuthology.orchestra.run.smithi139.stdout:795 write 0xac6bc00 thru 0xac72e79 (0x727a bytes) 2024-09-17T23:29:35.071 INFO:teuthology.orchestra.run.smithi139.stdout:796 read 0x2139800 thru 0x214317d (0x997e bytes) 2024-09-17T23:29:35.075 INFO:teuthology.orchestra.run.smithi139.stdout:797 write 0x80f8c00 thru 0x810326e (0xa66f bytes) 2024-09-17T23:29:35.075 INFO:teuthology.orchestra.run.smithi139.stdout:800 write 0x6bc9200 thru 0x6bd0127 (0x6f28 bytes) 2024-09-17T23:29:35.076 INFO:teuthology.orchestra.run.smithi139.stdout:803 read 0x3b85800 thru 0x3b8a50b (0x4d0c bytes) 2024-09-17T23:29:35.076 INFO:teuthology.orchestra.run.smithi139.stdout:804 read 0x90a3600 thru 0x90ab2c6 (0x7cc7 bytes) 2024-09-17T23:29:35.076 INFO:teuthology.orchestra.run.smithi139.stdout:806 write 0x774cc00 thru 0x775393a (0x6d3b bytes) 2024-09-17T23:29:35.077 INFO:teuthology.orchestra.run.smithi139.stdout:807 trunc from 0xac73000 to 0xa63a800 2024-09-17T23:29:35.107 INFO:teuthology.orchestra.run.smithi139.stdout:809 read 0x8060000 thru 0x806d2bd (0xd2be bytes) 2024-09-17T23:29:35.113 INFO:teuthology.orchestra.run.smithi139.stdout:810 punch from 0x620dc00 to 0x621b400, (0xd800 bytes) 2024-09-17T23:29:35.119 INFO:teuthology.orchestra.run.smithi139.stdout:811 trunc from 0xa63a800 to 0xe7dee00 2024-09-17T23:29:35.130 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:35.128+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:35.130 INFO:teuthology.orchestra.run.smithi139.stdout:812 read 0x7ebc200 thru 0x7ebc58f (0x390 bytes) 2024-09-17T23:29:35.137 INFO:teuthology.orchestra.run.smithi139.stdout:814 read 0x6508400 thru 0x6513ce0 (0xb8e1 bytes) 2024-09-17T23:29:35.137 INFO:teuthology.orchestra.run.smithi139.stdout:817 read 0x501200 thru 0x5021e1 (0xfe2 bytes) 2024-09-17T23:29:35.139 INFO:teuthology.orchestra.run.smithi139.stdout:823 read 0xe415000 thru 0xe41726b (0x226c bytes) 2024-09-17T23:29:35.139 INFO:teuthology.orchestra.run.smithi139.stdout:825 read 0xad98200 thru 0xad99ca5 (0x1aa6 bytes) 2024-09-17T23:29:35.139 INFO:teuthology.orchestra.run.smithi139.stdout:826 punch from 0x3eace00 to 0x3eb0800, (0x3a00 bytes) 2024-09-17T23:29:35.139 INFO:teuthology.orchestra.run.smithi139.stdout:827 punch from 0x78c000 to 0x795800, (0x9800 bytes) 2024-09-17T23:29:35.139 INFO:teuthology.orchestra.run.smithi139.stdout:829 read 0x88b0400 thru 0x88bfed1 (0xfad2 bytes) 2024-09-17T23:29:35.143 INFO:teuthology.orchestra.run.smithi139.stdout:830 punch from 0x8de6c00 to 0x8deda00, (0x6e00 bytes) 2024-09-17T23:29:35.143 INFO:teuthology.orchestra.run.smithi139.stdout:833 read 0x322de00 thru 0x3234bac (0x6dad bytes) 2024-09-17T23:29:35.143 INFO:teuthology.orchestra.run.smithi139.stdout:835 write 0x9e00800 thru 0x9e086ce (0x7ecf bytes) 2024-09-17T23:29:35.143 INFO:teuthology.orchestra.run.smithi139.stdout:836 read 0x36fc000 thru 0x37069a8 (0xa9a9 bytes) 2024-09-17T23:29:35.157 INFO:teuthology.orchestra.run.smithi139.stdout:838 trunc from 0xe7dee00 to 0x33e4600 2024-09-17T23:29:35.201 INFO:teuthology.orchestra.run.smithi139.stdout:839 write 0x6b4a00 thru 0x6b5557 (0xb58 bytes) 2024-09-17T23:29:35.201 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:35.202 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:35.207 INFO:teuthology.orchestra.run.smithi139.stdout:840 punch from 0x13ca00 to 0x141000, (0x4600 bytes) 2024-09-17T23:29:35.208 INFO:teuthology.orchestra.run.smithi139.stdout:841 read 0x726400 thru 0x72727c (0xe7d bytes) 2024-09-17T23:29:35.209 INFO:teuthology.orchestra.run.smithi139.stdout:842 punch from 0x2772200 to 0x2774000, (0x1e00 bytes) 2024-09-17T23:29:35.210 INFO:teuthology.orchestra.run.smithi139.stdout:844 read 0x2dccc00 thru 0x2dd9944 (0xcd45 bytes) 2024-09-17T23:29:35.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:35.213 INFO:teuthology.orchestra.run.smithi139.stdout:845 write 0x23af000 thru 0x23b418d (0x518e bytes) 2024-09-17T23:29:35.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:35.231 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:35.229+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:35.234 INFO:teuthology.orchestra.run.smithi139.stdout:847 write 0x5df9e00 thru 0x5e07668 (0xd869 bytes) 2024-09-17T23:29:35.238 INFO:teuthology.orchestra.run.smithi139.stdout:848 trunc from 0x5e07800 to 0x8b43200 2024-09-17T23:29:35.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:35.243 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:35.254 INFO:teuthology.orchestra.run.smithi139.stdout:849 write 0x4095e00 thru 0x40a01eb (0xa3ec bytes) 2024-09-17T23:29:35.261 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:35.261 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:35.279 INFO:teuthology.orchestra.run.smithi139.stdout:850 write 0xdc07800 thru 0xdc12396 (0xab97 bytes) 2024-09-17T23:29:35.280 INFO:teuthology.orchestra.run.smithi139.stdout:851 punch from 0x8ab9800 to 0x8abaa00, (0x1200 bytes) 2024-09-17T23:29:35.281 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:35.281 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:35.282 INFO:teuthology.orchestra.run.smithi139.stdout:852 punch from 0x67c0400 to 0x67c2c00, (0x2800 bytes) 2024-09-17T23:29:35.282 INFO:teuthology.orchestra.run.smithi139.stdout:856 punch from 0xd628400 to 0xd637800, (0xf400 bytes) 2024-09-17T23:29:35.283 INFO:teuthology.orchestra.run.smithi139.stdout:857 write 0x9c68800 thru 0x9c6e05b (0x585c bytes) 2024-09-17T23:29:35.283 INFO:teuthology.orchestra.run.smithi139.stdout:859 write 0x5f24c00 thru 0x5f2c75f (0x7b60 bytes) 2024-09-17T23:29:35.289 INFO:teuthology.orchestra.run.smithi139.stdout:860 trunc from 0xdc12400 to 0x7cb6c00 2024-09-17T23:29:35.305 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:35.312 INFO:teuthology.orchestra.run.smithi139.stdout:861 read 0x6819e00 thru 0x6828a75 (0xec76 bytes) 2024-09-17T23:29:35.318 INFO:teuthology.orchestra.run.smithi139.stdout:862 read 0x300de00 thru 0x30178b0 (0x9ab1 bytes) 2024-09-17T23:29:35.319 INFO:teuthology.orchestra.run.smithi139.stdout:863 write 0x5754800 thru 0x575c4cb (0x7ccc bytes) 2024-09-17T23:29:35.324 INFO:teuthology.orchestra.run.smithi139.stdout:864 trunc from 0x7cb6c00 to 0x3b30200 2024-09-17T23:29:35.325 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:35.325 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:35.334 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:35.331+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:35.344 INFO:teuthology.orchestra.run.smithi139.stdout:865 punch from 0x22afe00 to 0x22b9e00, (0xa000 bytes) 2024-09-17T23:29:35.348 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 35.874153505000002, 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:35.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1858 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.333070218, 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010693322999999999, 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1858 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012523+0000", 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.22e-07 2024-09-17T23:29:35.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012524+0000", 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1209999999999999e-06 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012527+0000", 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2490000000000001e-06 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012530+0000", 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5579999999999999e-06 2024-09-17T23:29:35.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023039+0000", 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010508671000000001 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023116+0000", 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7574e-05 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023216+0000", 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9927999999999996e-05 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1859 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.333023838000001, 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010887203, 2024-09-17T23:29:35.361 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1859 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012569+0000", 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012570+0000", 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012571+0000", 2024-09-17T23:29:35.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.75e-06 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012578+0000", 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8249999999999999e-06 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012581+0000", 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023262+0000", 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010680640999999999 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023344+0000", 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2785000000000001e-05 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023456+0000", 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111983 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1861 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.333007718999999, 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011093442, 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1861 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012585+0000", 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012586+0000", 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:35.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012590+0000", 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.724e-06 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012594+0000", 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6320000000000001e-06 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012597+0000", 2024-09-17T23:29:35.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6769999999999999e-06 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023495+0000", 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010898309 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023580+0000", 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.462e-05 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023679+0000", 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9230999999999999e-05 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1862 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332992944000001, 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011325251, 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1862 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012600+0000", 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0600000000000001e-07 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012602+0000", 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2589999999999999e-06 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012605+0000", 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.072e-06 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012607+0000", 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023725+0000", 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01111767 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023810+0000", 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840999999999996e-05 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023925+0000", 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011567500000000001 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1863 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332982866, 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011535393, 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1863 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012610+0000", 2024-09-17T23:29:35.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-07 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012612+0000", 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1799999999999999e-06 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012615+0000", 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012617+0000", 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5000000000000002e-06 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.023967+0000", 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011349704 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024045+0000", 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7955999999999997e-05 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024146+0000", 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100691 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1864 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 9830400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332972799, 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01177426, 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1864 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:35.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012620+0000", 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1400000000000001e-07 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012622+0000", 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-06 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012625+0000", 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.128e-06 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012627+0000", 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024192+0000", 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011564606999999999 2024-09-17T23:29:35.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024282+0000", 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9752000000000004e-05 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024395+0000", 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011288199999999999 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1865 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332962696999999, 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011984761999999999, 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1865 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012630+0000", 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012631+0000", 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012632+0000", 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.215e-06 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012635+0000", 2024-09-17T23:29:35.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1489999999999998e-06 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012637+0000", 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.57e-06 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024435+0000", 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011797368000000001 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024512+0000", 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7081000000000003e-05 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024615+0000", 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103176 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1866 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332952647999999, 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012220981000000001, 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1866 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012640+0000", 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012641+0000", 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:29:35.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012642+0000", 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1230000000000001e-06 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012645+0000", 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.18e-06 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012647+0000", 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4329999999999998e-06 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024660+0000", 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012012372 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024747+0000", 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7374999999999996e-05 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024861+0000", 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011429000000000001 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1867 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332942586, 2024-09-17T23:29:35.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012428603999999999, 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1867 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012650+0000", 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012651+0000", 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:35.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012652+0000", 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1969999999999999e-06 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012655+0000", 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1530000000000001e-06 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012658+0000", 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.543e-06 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024903+0000", 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012244994 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.024977+0000", 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3951999999999999e-05 2024-09-17T23:29:35.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025079+0000", 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102546 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1868 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:35.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332930709999999, 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012658264000000001, 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1868 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012662+0000", 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012663+0000", 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012664+0000", 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012667+0000", 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1839999999999999e-06 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012669+0000", 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4990000000000001e-06 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025123+0000", 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012453138000000001 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025207+0000", 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4153000000000006e-05 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025321+0000", 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113874 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1869 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332920495, 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012863466, 2024-09-17T23:29:35.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1869 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012673+0000", 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012674+0000", 2024-09-17T23:29:35.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.111e-06 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012677+0000", 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1190000000000001e-06 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012680+0000", 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5809999999999999e-06 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025360+0000", 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012680593 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025436+0000", 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5768999999999999e-05 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025536+0000", 2024-09-17T23:29:35.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100068 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1870 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332910406, 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013108685, 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1870 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012683+0000", 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012684+0000", 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0890000000000001e-06 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012687+0000", 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.146e-06 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012690+0000", 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5519999999999999e-06 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025589+0000", 2024-09-17T23:29:35.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012899713 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025676+0000", 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6457000000000002e-05 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025791+0000", 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115516 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1871 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.33290032, 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013256860000000001, 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1871 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012693+0000", 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1e-07 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012695+0000", 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.855e-06 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012698+0000", 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2600000000000001e-06 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012701+0000", 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025836+0000", 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013135338999999999 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025884+0000", 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7846999999999998e-05 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025950+0000", 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5791999999999996e-05 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1872 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332889195, 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013498906, 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1872 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012704+0000", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012705+0000", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2619999999999999e-06 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012709+0000", 2024-09-17T23:29:35.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1990000000000002e-06 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012711+0000", 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4310000000000001e-06 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.025996+0000", 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013284921 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026084+0000", 2024-09-17T23:29:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7754e-05 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026203+0000", 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119133 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1873 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:35.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332878997, 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013625369999999999, 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1873 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012714+0000", 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-07 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012723+0000", 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6079999999999996e-06 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012727+0000", 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0319999999999997e-06 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012730+0000", 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7590000000000002e-06 2024-09-17T23:29:35.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026236+0000", 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013505980000000001 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026279+0000", 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3838000000000003e-05 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026339+0000", 2024-09-17T23:29:35.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9953e-05 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1874 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 15073280~31744] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332859072, 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013877722, 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1874 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012734+0000", 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4699999999999998e-07 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012736+0000", 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3740000000000001e-06 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012739+0000", 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3019999999999999e-06 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012741+0000", 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5739999999999998e-06 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026384+0000", 2024-09-17T23:29:35.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013642965 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026465+0000", 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026612+0000", 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014634700000000001 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1875 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332848455000001, 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013997559, 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1875 2024-09-17T23:29:35.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012745+0000", 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-07 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012746+0000", 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2729999999999999e-06 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012749+0000", 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:29:35.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012752+0000", 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.672e-06 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026646+0000", 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013894098000000001 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026687+0000", 2024-09-17T23:29:35.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0540000000000001e-05 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026742+0000", 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5585999999999999e-05 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1876 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:35.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332838015, 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014222617999999999, 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1876 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012755+0000", 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3300000000000001e-07 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012757+0000", 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.353e-06 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012760+0000", 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1930000000000002e-06 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012762+0000", 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026784+0000", 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014021909000000001 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026867+0000", 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3014000000000002e-05 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.026978+0000", 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110282 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4486.0:1877 3.11 3:89ad46ff:::rbd_data.11522ab9a303.0000000000000003:4 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.332801197, 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014306071, 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4486", 2024-09-17T23:29:35.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1877 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012792+0000", 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3799999999999998e-07 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012794+0000", 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4169999999999999e-06 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012800+0000", 2024-09-17T23:29:35.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.896e-06 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.012803+0000", 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2760000000000001e-06 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027001+0000", 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014198555999999999 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027040+0000", 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8479000000000002e-05 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:23.027098+0000", 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8109000000000002e-05 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.444 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:35.446 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:35.444+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:35.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:867 trunc from 0x3b30200 to 0x767f000 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:868 write 0xcf95000 thru 0xcf9cf02 (0x7f03 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:869 write 0x628bc00 thru 0x628f494 (0x3895 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:871 trunc from 0xcf9d000 to 0xc94c600 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:874 punch from 0x4026800 to 0x4036600, (0xfe00 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:875 punch from 0xf5d400 to 0xf6a400, (0xd000 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:877 trunc from 0xc94c600 to 0x70d1000 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:879 punch from 0x18b5400 to 0x18b5c00, (0x800 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:880 punch from 0x95f200 to 0x961600, (0x2400 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:881 read 0x5eeda00 thru 0x5efcf45 (0xf546 bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:882 write 0x9170400 thru 0x917739c (0x6f9d bytes) 2024-09-17T23:29:35.448 INFO:teuthology.orchestra.run.smithi139.stdout:883 punch from 0x4ca1c00 to 0x4ca9800, (0x7c00 bytes) 2024-09-17T23:29:35.449 INFO:teuthology.orchestra.run.smithi139.stdout:885 read 0x4cfde00 thru 0x4d02cde (0x4edf bytes) 2024-09-17T23:29:35.449 INFO:teuthology.orchestra.run.smithi139.stdout:886 read 0x1bdaa00 thru 0x1be46bb (0x9cbc bytes) 2024-09-17T23:29:35.449 INFO:teuthology.orchestra.run.smithi139.stdout:887 read 0x60c0800 thru 0x60ce53b (0xdd3c bytes) 2024-09-17T23:29:35.458 INFO:teuthology.orchestra.run.smithi139.stdout:889 trunc from 0x9177400 to 0x1b1ba00 2024-09-17T23:29:35.508 INFO:teuthology.orchestra.run.smithi139.stdout:890 write 0x7459e00 thru 0x74673fd (0xd5fe bytes) 2024-09-17T23:29:35.516 INFO:teuthology.orchestra.run.smithi139.stdout:892 trunc from 0x7467400 to 0xdc2ce00 2024-09-17T23:29:35.528 INFO:teuthology.orchestra.run.smithi139.stdout:893 read 0x1d2ee00 thru 0x1d2fac7 (0xcc8 bytes) 2024-09-17T23:29:35.536 INFO:teuthology.orchestra.run.smithi139.stdout:897 read 0x335a600 thru 0x335bc6e (0x166f bytes) 2024-09-17T23:29:35.536 INFO:teuthology.orchestra.run.smithi139.stdout:899 write 0x76bce00 thru 0x76cb302 (0xe503 bytes) 2024-09-17T23:29:35.536 INFO:teuthology.orchestra.run.smithi139.stdout:900 punch from 0xc277400 to 0xc27c800, (0x5400 bytes) 2024-09-17T23:29:35.538 INFO:teuthology.orchestra.run.smithi139.stdout:901 read 0x1a00c00 thru 0x1a08ada (0x7edb bytes) 2024-09-17T23:29:35.538 INFO:teuthology.orchestra.run.smithi139.stdout:902 write 0x8c78e00 thru 0x8c83175 (0xa376 bytes) 2024-09-17T23:29:35.539 INFO:teuthology.orchestra.run.smithi139.stdout:905 read 0x82ce000 thru 0x82d0bc0 (0x2bc1 bytes) 2024-09-17T23:29:35.539 INFO:teuthology.orchestra.run.smithi139.stdout:906 punch from 0x5308000 to 0x5316a00, (0xea00 bytes) 2024-09-17T23:29:35.547 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:35.545+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:35.548 INFO:teuthology.orchestra.run.smithi139.stdout:907 trunc from 0xdc2ce00 to 0x5979600 2024-09-17T23:29:35.562 INFO:teuthology.orchestra.run.smithi139.stdout:910 read 0xb57800 thru 0xb66eb9 (0xf6ba bytes) 2024-09-17T23:29:35.569 INFO:teuthology.orchestra.run.smithi139.stdout:912 trunc from 0x5979600 to 0xa0b4400 2024-09-17T23:29:35.577 INFO:teuthology.orchestra.run.smithi139.stdout:913 punch from 0x7b7cc00 to 0x7b7fe00, (0x3200 bytes) 2024-09-17T23:29:35.585 INFO:teuthology.orchestra.run.smithi139.stdout:915 read 0x7f6ec00 thru 0x7f7b4e4 (0xc8e5 bytes) 2024-09-17T23:29:35.592 INFO:teuthology.orchestra.run.smithi139.stdout:916 trunc from 0xa0b4400 to 0x45af600 2024-09-17T23:29:35.605 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:35.606 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:35.606 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:35.606 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:35.608 INFO:teuthology.orchestra.run.smithi139.stdout:920 trunc from 0x45af600 to 0x5d3b200 2024-09-17T23:29:35.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:35.629 INFO:teuthology.orchestra.run.smithi139.stdout:924 punch from 0x40e9600 to 0x40eee00, (0x5800 bytes) 2024-09-17T23:29:35.629 INFO:teuthology.orchestra.run.smithi139.stdout:927 read 0x1b50e00 thru 0x1b55714 (0x4915 bytes) 2024-09-17T23:29:35.630 INFO:teuthology.orchestra.run.smithi139.stdout:928 flatten 2024-09-17T23:29:35.657 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:35.655+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:35.657 INFO:teuthology.orchestra.run.smithi139.stdout:931 write 0x1dd0800 thru 0x1dd60a2 (0x58a3 bytes) 2024-09-17T23:29:35.659 INFO:teuthology.orchestra.run.smithi139.stdout:933 read 0x2994600 thru 0x299c98f (0x8390 bytes) 2024-09-17T23:29:35.659 INFO:teuthology.orchestra.run.smithi139.stdout:935 punch from 0x2200400 to 0x2201600, (0x1200 bytes) 2024-09-17T23:29:35.659 INFO:teuthology.orchestra.run.smithi139.stdout:937 punch from 0x59b4200 to 0x59b5c00, (0x1a00 bytes) 2024-09-17T23:29:35.659 INFO:teuthology.orchestra.run.smithi139.stdout:939 read 0x11fe600 thru 0x120bc69 (0xd66a bytes) 2024-09-17T23:29:35.659 INFO:teuthology.orchestra.run.smithi139.stdout:940 punch from 0x5bc8c00 to 0x5bd6e00, (0xe200 bytes) 2024-09-17T23:29:35.660 INFO:teuthology.orchestra.run.smithi139.stdout:941 read 0x2361400 thru 0x2365122 (0x3d23 bytes) 2024-09-17T23:29:35.660 INFO:teuthology.orchestra.run.smithi139.stdout:942 punch from 0xf87e00 to 0xf8fa00, (0x7c00 bytes) 2024-09-17T23:29:35.660 INFO:teuthology.orchestra.run.smithi139.stdout:943 punch from 0x2335e00 to 0x233a600, (0x4800 bytes) 2024-09-17T23:29:35.666 INFO:teuthology.orchestra.run.smithi139.stdout:944 trunc from 0x5d3b200 to 0xb5a6e00 2024-09-17T23:29:35.679 INFO:teuthology.orchestra.run.smithi139.stdout:945 read 0xb59e600 thru 0xb5a6c15 (0x8616 bytes) 2024-09-17T23:29:35.690 INFO:teuthology.orchestra.run.smithi139.stdout:948 write 0xb852800 thru 0xb859478 (0x6c79 bytes) 2024-09-17T23:29:35.691 INFO:teuthology.orchestra.run.smithi139.stdout:951 write 0x7958200 thru 0x7967f29 (0xfd2a bytes) 2024-09-17T23:29:35.691 INFO:teuthology.orchestra.run.smithi139.stdout:953 read 0xf7600 thru 0xfb61e (0x401f bytes) 2024-09-17T23:29:35.691 INFO:teuthology.orchestra.run.smithi139.stdout:954 write 0x60ebe00 thru 0x60f0610 (0x4811 bytes) 2024-09-17T23:29:35.691 INFO:teuthology.orchestra.run.smithi139.stdout:955 write 0x620e200 thru 0x621859a (0xa39b bytes) 2024-09-17T23:29:35.692 INFO:teuthology.orchestra.run.smithi139.stdout:956 write 0x3d81c00 thru 0x3d86c11 (0x5012 bytes) 2024-09-17T23:29:35.692 INFO:teuthology.orchestra.run.smithi139.stdout:958 read 0x49c3c00 thru 0x49cb9a7 (0x7da8 bytes) 2024-09-17T23:29:35.692 INFO:teuthology.orchestra.run.smithi139.stdout:961 punch from 0x517f800 to 0x5182c00, (0x3400 bytes) 2024-09-17T23:29:35.692 INFO:teuthology.orchestra.run.smithi139.stdout:963 punch from 0x8db3000 to 0x8db4800, (0x1800 bytes) 2024-09-17T23:29:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:966 read 0x504800 thru 0x508323 (0x3b24 bytes) 2024-09-17T23:29:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:967 write 0x9e46000 thru 0x9e4cfdb (0x6fdc bytes) 2024-09-17T23:29:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:968 write 0xa189a00 thru 0xa191ea6 (0x84a7 bytes) 2024-09-17T23:29:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:969 punch from 0x487f400 to 0x4883000, (0x3c00 bytes) 2024-09-17T23:29:35.706 INFO:teuthology.orchestra.run.smithi139.stdout:970 trunc from 0xb859600 to 0xa10800 2024-09-17T23:29:35.741 INFO:teuthology.orchestra.run.smithi139.stdout:971 punch from 0x127600 to 0x130c00, (0x9600 bytes) 2024-09-17T23:29:35.741 INFO:teuthology.orchestra.run.smithi139.stdout:973 read 0x865600 thru 0x871e3c (0xc83d bytes) 2024-09-17T23:29:35.743 INFO:teuthology.orchestra.run.smithi139.stdout:974 read 0xa6000 thru 0xa8a7e (0x2a7f bytes) 2024-09-17T23:29:35.743 INFO:teuthology.orchestra.run.smithi139.stdout:975 punch from 0x505600 to 0x510e00, (0xb800 bytes) 2024-09-17T23:29:35.744 INFO:teuthology.orchestra.run.smithi139.stdout:976 clone 8 order 22 su 1048576 sc 3 2024-09-17T23:29:35.748 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:35.746+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:35.778 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:35.778 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:35.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:35.871 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:35.869+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:35.937 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:35.937 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4236.0:1 2.3 e16/15 2:c4c92e5a:::rbd_trash:head v 16'1, mlcod=0'0)", 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.433933919000001, 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003809191, 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499863+0000", 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-17T23:29:35.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499873+0000", 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.91e-06 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499897+0000", 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4607999999999999e-05 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499912+0000", 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.435e-05 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.499975+0000", 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2942999999999997e-05 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.500010+0000", 2024-09-17T23:29:35.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4839000000000003e-05 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503630+0000", 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003619911 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:05.503672+0000", 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2315999999999999e-05 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4481.0:393 3.1d 3:b818411f:::rbd_data.11521d8bcd2d.0000000000000005:head [read 12496896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.914491+0000", 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0193057759999995, 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021747030000000001, 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4481", 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 393 2024-09-17T23:29:35.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914491+0000", 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914491+0000", 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914492+0000", 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.646e-06 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914497+0000", 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8289999999999998e-06 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914507+0000", 2024-09-17T23:29:35.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5489999999999995e-06 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914514+0000", 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1019999999999999e-06 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914556+0000", 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1613000000000003e-05 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for missing object", 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.914652+0000", 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6348000000000007e-05 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.916475+0000", 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018231600000000001 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.916556+0000", 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0772000000000005e-05 2024-09-17T23:29:35.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.916666+0000", 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109684 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4481.0:453 3.1d 3:b818411f:::rbd_data.11521d8bcd2d.0000000000000005:head [stat,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.933738+0000", 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0000586630000008, 2024-09-17T23:29:35.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027380360000000001, 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4481", 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 453 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.933738+0000", 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.933738+0000", 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.933739+0000", 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1489999999999999e-06 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.933745+0000", 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8050000000000003e-06 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.933756+0000", 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.933765+0000", 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3559999999999999e-06 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934286+0000", 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052022500000000003 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934387+0000", 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101631 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934537+0000", 2024-09-17T23:29:35.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014939599999999999 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936063+0000", 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015259 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936221+0000", 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000158777 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936288+0000", 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6771000000000002e-05 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936476+0000", 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018783499999999999 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4481.0:454 3.1d 3:b818411f:::rbd_data.11521d8bcd2d.0000000000000005:head [read 12451840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.934458+0000", 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9993390590000004, 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022329479999999998, 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4481", 2024-09-17T23:29:35.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 454 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934458+0000", 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934458+0000", 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934458+0000", 2024-09-17T23:29:35.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8400000000000003e-07 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934463+0000", 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.527e-06 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934479+0000", 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6419999999999999e-05 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934487+0000", 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8010000000000004e-06 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934788+0000", 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030066700000000001 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for rw locks", 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934833+0000", 2024-09-17T23:29:35.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5596999999999997e-05 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936528+0000", 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001694709 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936587+0000", 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000002e-05 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936690+0000", 2024-09-17T23:29:35.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010331300000000001 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4481.0:455 3.1d 3:b818411f:::rbd_data.11521d8bcd2d.0000000000000005:head [read 12460032~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.934499+0000", 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9992972230000001, 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025703710000000001, 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4481", 2024-09-17T23:29:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 455 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934499+0000", 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934499+0000", 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934500+0000", 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2300000000000002e-07 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934502+0000", 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0949999999999998e-06 2024-09-17T23:29:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934508+0000", 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0680000000000002e-06 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934512+0000", 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7799999999999998e-06 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934882+0000", 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037053800000000001 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for rw locks", 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934978+0000", 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6087000000000003e-05 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936746+0000", 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017679950000000001 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936848+0000", 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101809 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.937070+0000", 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022157599999999999 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4481.0:456 3.1d 3:b818411f:::rbd_data.11521d8bcd2d.0000000000000005:head [sparse-read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.934525+0000", 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9992711459999999, 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002776948, 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4481", 2024-09-17T23:29:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 456 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934525+0000", 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934525+0000", 2024-09-17T23:29:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934526+0000", 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4299999999999999e-07 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934528+0000", 2024-09-17T23:29:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.599e-06 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934534+0000", 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5879999999999997e-06 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.934538+0000", 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5399999999999997e-06 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.935011+0000", 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000472793 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for rw locks", 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.935085+0000", 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3935000000000003e-05 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.937140+0000", 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020549729999999999 2024-09-17T23:29:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.937188+0000", 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7713000000000002e-05 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.937302+0000", 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011446399999999999 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4481.0:438 3.8 e31/30 3:123517d2:::rbd_data.11521d8bcd2d.0000000000000001:head v 31'18, mlcod=0'0)", 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.936947+0000", 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9968495409999996, 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046300969999999997, 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936947+0000", 2024-09-17T23:29:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936947+0000", 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936948+0000", 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2699999999999999e-07 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936953+0000", 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9620000000000001e-06 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936960+0000", 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1899999999999998e-06 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.936964+0000", 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1819999999999999e-06 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.937012+0000", 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8288999999999998e-05 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.937041+0000", 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8901e-05 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.941533+0000", 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044919640000000002 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.941577+0000", 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3982e-05 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "MOSDPGPush(3.1e 31/30 [PushOp(3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head, version: 29'18, data_included: [1048576~262144,1355264~42496,1441792~743936], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head@29'18, size: 11008512, copy_subset: [(0, 11008512)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2185728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T23:29:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.983604+0000", 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9501921299999996, 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024076179999999998, 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.983604+0000", 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.983604+0000", 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.983605+0000", 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2299999999999999e-07 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.984536+0000", 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093079600000000001 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.984559+0000", 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2532e-05 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.984578+0000", 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8901000000000001e-05 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.984668+0000", 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0903999999999996e-05 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.984685+0000", 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6410999999999999e-05 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.986012+0000", 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013271509999999999 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "MOSDPGPush(3.1e 31/30 [PushOp(3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head, version: 29'18, data_included: [3365376~828928,8256000~55296,10957824~50688], data_size: 934912, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head@29'18, size: 11008512, copy_subset: [(0, 11008512)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 11008512, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3365376, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:27.991478+0000", 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.942318105, 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002299518, 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.991478+0000", 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.991478+0000", 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.991479+0000", 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5599999999999995e-07 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.992368+0000", 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00088849100000000004 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.992385+0000", 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7192000000000001e-05 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.992395+0000", 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0581000000000001e-05 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.992487+0000", 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2158999999999996e-05 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.992503+0000", 2024-09-17T23:29:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5191000000000001e-05 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:27.993778+0000", 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012753479999999999 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4623.0:61 3.1e e34/30 3:7efb4fc6:::rbd_object_map.1152d63cf230:head v 34'33, mlcod=33'31)", 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:30.921960+0000", 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.01183625, 2024-09-17T23:29:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028657909999999999, 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.921960+0000", 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.921960+0000", 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.921961+0000", 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2999999999999999e-07 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.921967+0000", 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.921978+0000", 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1535000000000001e-05 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.921988+0000", 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4890000000000001e-06 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.922042+0000", 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4370999999999998e-05 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.922066+0000", 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3969999999999999e-05 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.924777+0000", 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027109180000000001 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.924826+0000", 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9211e-05 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4623.0:115 3.1 e34/33 3:819959cd:::rbd_data.1152d63cf230.0000000000000006:head v 34'136, mlcod=0'0)", 2024-09-17T23:29:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:30.937521+0000", 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9962756769999999, 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031205909999999998, 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937521+0000", 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937521+0000", 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937522+0000", 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0299999999999998e-07 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937554+0000", 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2026000000000001e-05 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937571+0000", 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7742000000000001e-05 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937578+0000", 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9469999999999997e-06 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937633+0000", 2024-09-17T23:29:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4361999999999999e-05 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.937648+0000", 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5135e-05 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.940605+0000", 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029573199999999998 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.940641+0000", 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6356000000000002e-05 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4623.0:180 3.5 3:a24b2366:::rbd_data.1152d63cf230.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 8297472~46592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:30.958766+0000", 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9750309589999997, 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025994820000000002, 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4623", 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2012153814", 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 180 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958766+0000", 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958766+0000", 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958767+0000", 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958904+0000", 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013666000000000001 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958914+0000", 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9040000000000004e-06 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958919+0000", 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958969+0000", 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0269999999999998e-05 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.959130+0000", 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016043599999999999 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.959371+0000", 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000241547 2024-09-17T23:29:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.960599+0000", 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00122773 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.961296+0000", 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00069663699999999995 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.961331+0000", 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4678999999999998e-05 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.961365+0000", 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4572e-05 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4623.0:181 3.16 3:6ac8cf5e:::rbd_data.1152d63cf230.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 3966976~15360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T23:29:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:30.958934+0000", 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.974862323, 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042831019999999996, 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4623", 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2012153814", 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 181 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958934+0000", 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958934+0000", 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958935+0000", 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2399999999999998e-07 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958946+0000", 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1438e-05 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958952+0000", 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5999999999999997e-06 2024-09-17T23:29:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.958955+0000", 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4379999999999999e-06 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.959000+0000", 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5033000000000001e-05 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.959127+0000", 2024-09-17T23:29:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012718399999999999 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.959372+0000", 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000244066 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.960600+0000", 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012283229999999999 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.963159+0000", 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002559141 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.963195+0000", 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.578e-05 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.963217+0000", 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2575e-05 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4623.0:241 3.1e e34/30 3:7efb4fc6:::rbd_object_map.1152d63cf230:head v 34'41, mlcod=34'38)", 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:30.979732+0000", 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9540645430000003, 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032867450000000002, 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979732+0000", 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979732+0000", 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979733+0000", 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2399999999999998e-07 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979738+0000", 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979748+0000", 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8009999999999998e-06 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979756+0000", 2024-09-17T23:29:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380000000000005e-06 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979791+0000", 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4937000000000001e-05 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.979803+0000", 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1912e-05 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.982971+0000", 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031685250000000002 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:30.983019+0000", 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7395000000000003e-05 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4683.0:1772 3.14 e37/30 3:2c0fb067:::rbd_data.115230678ec7.000000000000005e:head v 37'16, mlcod=37'14)", 2024-09-17T23:29:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:35.292693+0000", 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.64110366100000005, 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002276223, 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292693+0000", 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292693+0000", 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292694+0000", 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5600000000000001e-06 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292735+0000", 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0640999999999999e-05 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292755+0000", 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9638000000000001e-05 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292765+0000", 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031e-05 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292829+0000", 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3808999999999998e-05 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.292853+0000", 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-05 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.294924+0000", 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020716749999999998 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.294969+0000", 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4669999999999998e-05 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4683.0:2313 3.2 e37/30 3:4118c975:::rbd_data.115230678ec7.000000000000001e:head v 37'16, mlcod=37'14)", 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:35.462486+0000", 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.47131050099999999, 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022367070000000001, 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462486+0000", 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462486+0000", 2024-09-17T23:29:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462487+0000", 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0100000000000004e-07 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462494+0000", 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7780000000000002e-06 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462505+0000", 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1136e-05 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462512+0000", 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0609999999999998e-06 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462571+0000", 2024-09-17T23:29:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9639000000000003e-05 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.462592+0000", 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1092000000000001e-05 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464685+0000", 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002092652 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464723+0000", 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7648000000000003e-05 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4683.0:2336 3.16 e37/36 3:689e0c23:::rbd_data.115230678ec7.0000000000000037:head v 37'49, mlcod=37'47)", 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:35.464367+0000", 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.46942943999999998, 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002690773, 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464367+0000", 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464367+0000", 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464368+0000", 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464373+0000", 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3329999999999996e-06 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464384+0000", 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0898e-05 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464390+0000", 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2829999999999998e-06 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464450+0000", 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0007000000000003e-05 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.464473+0000", 2024-09-17T23:29:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2932e-05 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.467016+0000", 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025429739999999999 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.467058+0000", 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1729e-05 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4683.0:2612 3.11 e37/36 3:8aace9dd:::rbd_data.115230678ec7.0000000000000052:head v 37'23, mlcod=37'21)", 2024-09-17T23:29:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:35.545473+0000", 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.38832348799999999, 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002366361, 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545473+0000", 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545473+0000", 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545474+0000", 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545503+0000", 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8547999999999998e-05 2024-09-17T23:29:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545516+0000", 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3762e-05 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545524+0000", 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4749999999999996e-06 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545584+0000", 2024-09-17T23:29:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0210999999999997e-05 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545609+0000", 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4629000000000001e-05 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.547790+0000", 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021809799999999999 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.547839+0000", 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9663999999999997e-05 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4683.0:2613 3.e e37/36 3:77492d15:::rbd_data.115230678ec7.0000000000000053:head v 37'12, mlcod=37'10)", 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:35.545658+0000", 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.38813880299999998, 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023971029999999998, 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545658+0000", 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545658+0000", 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545659+0000", 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9400000000000004e-07 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545683+0000", 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4105e-05 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545692+0000", 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1640000000000007e-06 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545698+0000", 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1079999999999998e-06 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545750+0000", 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1872000000000001e-05 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.545770+0000", 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0069e-05 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.548013+0000", 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002242642 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.548055+0000", 2024-09-17T23:29:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2348999999999997e-05 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4683.0:3211 3.16 e37/36 3:6d9d3377:::rbd_data.115230678ec7.0000000000000079:head v 37'55, mlcod=37'53)", 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:35.700956+0000", 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.23284079999999999, 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00304142, 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.700956+0000", 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.700956+0000", 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.700957+0000", 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.17e-06 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.700979+0000", 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2419999999999999e-05 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.700996+0000", 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7096999999999999e-05 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.701004+0000", 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5469999999999999e-06 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.701064+0000", 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9735000000000003e-05 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.701086+0000", 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.19e-05 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.703954+0000", 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028682220000000001 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:35.703997+0000", 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3328999999999998e-05 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:36.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:29:36.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:36.054 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:36.051+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:36.154 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:36.152+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:36.163 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:36.163 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:36.163 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:36.163 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:36.173 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:36.252 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T23:29:36.254 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:36.253+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:36.280 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:36.281 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:36.281 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:36.281 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:36.281 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:36.289 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:36.312 INFO:teuthology.orchestra.run.smithi139.stdout:979 trunc from 0xa10800 to 0x4080200 2024-09-17T23:29:36.325 INFO:teuthology.orchestra.run.smithi139.stdout:982 read 0x240d400 thru 0x241b506 (0xe107 bytes) 2024-09-17T23:29:36.326 INFO:teuthology.orchestra.run.smithi139.stdout:985 punch from 0x253c600 to 0x2543400, (0x6e00 bytes) 2024-09-17T23:29:36.352 INFO:teuthology.orchestra.run.smithi139.stdout:987 write 0xea05600 thru 0xea05e2c (0x82d bytes) 2024-09-17T23:29:36.352 INFO:teuthology.orchestra.run.smithi139.stdout:988 read 0x965e00 thru 0x96f3bb (0x95bc bytes) 2024-09-17T23:29:36.357 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:36.355+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:36.358 INFO:teuthology.orchestra.run.smithi139.stdout:995 write 0x9bcbc00 thru 0x9bd63aa (0xa7ab bytes) 2024-09-17T23:29:36.358 INFO:teuthology.orchestra.run.smithi139.stdout:998 read 0x4729400 thru 0x4732bd4 (0x97d5 bytes) 2024-09-17T23:29:36.358 INFO:teuthology.orchestra.run.smithi139.stdout:999 punch from 0x51f8e00 to 0x51fb800, (0x2a00 bytes) 2024-09-17T23:29:36.358 INFO:teuthology.orchestra.run.smithi139.stdout:1000 punch from 0x1888a00 to 0x1897400, (0xea00 bytes) 2024-09-17T23:29:36.362 INFO:teuthology.orchestra.run.smithi139.stdout:1001 trunc from 0xea06000 to 0xae54e00 2024-09-17T23:29:36.392 INFO:teuthology.orchestra.run.smithi139.stdout:1002 trunc from 0xae54e00 to 0x3ddbc00 2024-09-17T23:29:36.405 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:36.405 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:36.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:36.405 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:36.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.405 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1530 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 26083328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430997648, 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013083955, 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1530 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:36.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971182+0000", 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2999999999999999e-07 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971183+0000", 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1299999999999998e-07 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971186+0000", 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0290000000000001e-06 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971188+0000", 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4279999999999999e-06 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984090+0000", 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.012901151 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984168+0000", 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8838000000000006e-05 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984266+0000", 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7366000000000003e-05 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.410 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1531 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 26607616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971213+0000", 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430966029, 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013149779, 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1531 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971213+0000", 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971213+0000", 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971214+0000", 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6300000000000001e-07 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971215+0000", 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.528e-06 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971219+0000", 2024-09-17T23:29:36.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.557e-06 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971221+0000", 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5239999999999999e-06 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984278+0000", 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013056885000000001 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984312+0000", 2024-09-17T23:29:36.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3776999999999999e-05 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984363+0000", 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1144999999999999e-05 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1532 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 27131904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430954806000001, 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013345258, 2024-09-17T23:29:36.415 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1532 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971225+0000", 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0599999999999999e-07 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971226+0000", 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0589999999999999e-06 2024-09-17T23:29:36.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971229+0000", 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0589999999999998e-06 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971231+0000", 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.446e-06 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984396+0000", 2024-09-17T23:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013164110999999999 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984474+0000", 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8658000000000005e-05 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984570+0000", 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5718999999999996e-05 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.419 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1533 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 27656192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971234+0000", 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.43094518, 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013433272, 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.420 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1533 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971234+0000", 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971234+0000", 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971235+0000", 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2300000000000002e-07 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971236+0000", 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971239+0000", 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9979999999999999e-06 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971241+0000", 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984582+0000", 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013340604000000001 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984617+0000", 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5660000000000001e-05 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984668+0000", 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0266999999999998e-05 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.424 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1534 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 28180480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430935636999999, 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.01363252, 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1534 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971244+0000", 2024-09-17T23:29:36.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1e-07 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971249+0000", 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8239999999999999e-06 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971255+0000", 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971259+0000", 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3900000000000003e-06 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984701+0000", 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0134419 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984782+0000", 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0703000000000003e-05 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984876+0000", 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4767999999999998e-05 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1535 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 28704768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.429 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971265+0000", 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430914049, 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013710455999999999, 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1535 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971265+0000", 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971265+0000", 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971266+0000", 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.13e-07 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971268+0000", 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.933e-06 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971274+0000", 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2230000000000004e-06 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971279+0000", 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1200000000000001e-06 2024-09-17T23:29:36.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984889+0000", 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013609903 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984926+0000", 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6628000000000002e-05 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.984976+0000", 2024-09-17T23:29:36.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0136000000000002e-05 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1536 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 29229056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430893858999999, 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013896568, 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.434 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1536 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971286+0000", 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.89e-07 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971288+0000", 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.84e-06 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971294+0000", 2024-09-17T23:29:36.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.489e-06 2024-09-17T23:29:36.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971299+0000", 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9200000000000003e-06 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985010+0000", 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013711008 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985087+0000", 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6694999999999997e-05 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985182+0000", 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5227000000000002e-05 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1537 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 29753344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430869849, 2024-09-17T23:29:36.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013987474999999999, 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1537 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:36.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971310+0000", 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5300000000000004e-07 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971312+0000", 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9149999999999999e-06 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971321+0000", 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5380000000000002e-06 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971334+0000", 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2962000000000001e-05 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985209+0000", 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013875458 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985247+0000", 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7829e-05 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985297+0000", 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0219999999999997e-05 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.443 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1538 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 30277632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430816893999999, 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014136910000000001, 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.444 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1538 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971363+0000", 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000004e-07 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971366+0000", 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5440000000000001e-06 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971374+0000", 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0190000000000006e-06 2024-09-17T23:29:36.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971380+0000", 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8209999999999998e-06 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985329+0000", 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.013949088 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985405+0000", 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5956000000000003e-05 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985500+0000", 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4812e-05 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1539 3.1e 3:7b382d84:::rbd_data.11522ab9a303.0000000000000007:4 [sparse-read 30801920~64000] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:36.448 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.430791878000001, 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014374316, 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1539 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971388+0000", 2024-09-17T23:29:36.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8999999999999997e-07 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971396+0000", 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7740000000000004e-06 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971403+0000", 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2459999999999997e-06 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.971409+0000", 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4779999999999999e-06 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985512+0000", 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014103861000000001 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985546+0000", 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3210000000000002e-05 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.985762+0000", 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000216257 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:157 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 8256000~55296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.810463+0000", 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5917163619999997, 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.015970193000000001, 2024-09-17T23:29:36.453 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 157 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.810463+0000", 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.810463+0000", 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.810464+0000", 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2699999999999998e-07 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818308+0000", 2024-09-17T23:29:36.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078437009999999998 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818322+0000", 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4761e-05 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818329+0000", 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6109999999999999e-06 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818400+0000", 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1370000000000003e-05 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818497+0000", 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6812999999999996e-05 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.818726+0000", 2024-09-17T23:29:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000228748 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.823365+0000", 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046387970000000001 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.826361+0000", 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00299606 2024-09-17T23:29:36.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.826400+0000", 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8729000000000001e-05 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.826433+0000", 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3776000000000003e-05 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5274332200000007, 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.460 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5118080900000006, 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:29:36.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.473 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5117704340000007, 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:29:36.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.478 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5117530030000008, 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:29:36.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5117180720000007, 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.485 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:36.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:29:36.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:29:36.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:29:36.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:29:36.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:36.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:29:36.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:29:36.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:29:36.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:29:36.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:29:36.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:29:36.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:29:36.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:29:36.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.494 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5116866519999999, 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:29:36.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5116700260000009, 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:36.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:29:36.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:29:36.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:29:36.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:29:36.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.511648332, 2024-09-17T23:29:36.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:29:36.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:29:36.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5116366140000004, 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:29:36.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.522 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:36.525 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:36.525 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:36.521+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1004 write 0x8890a00 thru 0x889f69a (0xec9b bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1005 write 0xdef1a00 thru 0xdefc355 (0xa956 bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1010 trunc from 0xdefc400 to 0x95cb400 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1012 read 0x463e800 thru 0x463f37f (0xb80 bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1013 punch from 0x2948400 to 0x294ce00, (0x4a00 bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1015 write 0xccc000 thru 0xcd1432 (0x5433 bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1018 write 0xc53e200 thru 0xc543f4c (0x5d4d bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1019 read 0x70c6600 thru 0x70cbd8b (0x578c bytes) 2024-09-17T23:29:36.526 INFO:teuthology.orchestra.run.smithi139.stdout:1021 trunc from 0xc544000 to 0x1570800 2024-09-17T23:29:36.532 INFO:teuthology.orchestra.run.smithi139.stdout:1025 punch from 0x128aa00 to 0x128e400, (0x3a00 bytes) 2024-09-17T23:29:36.532 INFO:teuthology.orchestra.run.smithi139.stdout:1026 read 0x145d400 thru 0x14627ce (0x53cf bytes) 2024-09-17T23:29:36.532 INFO:teuthology.orchestra.run.smithi139.stdout:1028 read 0xe06e00 thru 0xe0b3bc (0x45bd bytes) 2024-09-17T23:29:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:1029 write 0x4957c00 thru 0x4962c21 (0xb022 bytes) 2024-09-17T23:29:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:1032 read 0x139c600 thru 0x139c6e9 (0xea bytes) 2024-09-17T23:29:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:1034 read 0x3e6ae00 thru 0x3e7310f (0x8310 bytes) 2024-09-17T23:29:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:1035 write 0xe5b000 thru 0xe681ec (0xd1ed bytes) 2024-09-17T23:29:36.552 INFO:teuthology.orchestra.run.smithi139.stdout:1037 write 0xb60000 thru 0xb61a58 (0x1a59 bytes) 2024-09-17T23:29:36.553 INFO:teuthology.orchestra.run.smithi139.stdout:1038 trunc from 0x4962e00 to 0x5298800 2024-09-17T23:29:36.583 INFO:teuthology.orchestra.run.smithi139.stdout:1040 trunc from 0x5298800 to 0xde3de00 2024-09-17T23:29:36.593 INFO:teuthology.orchestra.run.smithi139.stdout:1041 read 0x73e2800 thru 0x73e61ed (0x39ee bytes) 2024-09-17T23:29:36.594 INFO:teuthology.orchestra.run.smithi139.stdout:1042 read 0xca9e00 thru 0xcb3241 (0x9442 bytes) 2024-09-17T23:29:36.606 INFO:teuthology.orchestra.run.smithi139.stdout:1044 trunc from 0xde3de00 to 0x7667200 2024-09-17T23:29:36.620 INFO:teuthology.orchestra.run.smithi139.stdout:1046 trunc from 0x7667200 to 0xb6e2400 2024-09-17T23:29:36.624 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:36.623+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:36.628 INFO:teuthology.orchestra.run.smithi139.stdout:1048 read 0x4042200 thru 0x404e98d (0xc78e bytes) 2024-09-17T23:29:36.628 INFO:teuthology.orchestra.run.smithi139.stdout:1049 write 0x3bffa00 thru 0x3c006b2 (0xcb3 bytes) 2024-09-17T23:29:36.639 INFO:teuthology.orchestra.run.smithi139.stdout:1051 trunc from 0xb6e2400 to 0x2723c00 2024-09-17T23:29:36.655 INFO:teuthology.orchestra.run.smithi139.stdout:1052 clone 9 order 18 su 1024 sc 9 2024-09-17T23:29:36.681 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:36.681 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:36.689 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:36.733 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:36.731+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:36.796 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:36.796 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:36.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:36.826 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:36.824+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:36.927 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:36.926+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:36.947 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:477 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:36.956 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.84004547, 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081640670000000005, 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 477 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112792+0000", 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112793+0000", 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2e-07 2024-09-17T23:29:36.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112796+0000", 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9620000000000002e-06 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112803+0000", 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5989999999999996e-06 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112809+0000", 2024-09-17T23:29:36.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.276e-06 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117872+0000", 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050631349999999999 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.117960+0000", 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8399000000000005e-05 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:36.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.118168+0000", 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020731300000000001 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120846+0000", 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026780630000000001 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120889+0000", 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2904000000000001e-05 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120920+0000", 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1075999999999999e-05 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120956+0000", 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6820000000000003e-05 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:478 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.840010798000002, 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0088678699999999999, 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 478 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112827+0000", 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2599999999999998e-07 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112831+0000", 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1669999999999999e-06 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112839+0000", 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4230000000000005e-06 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112845+0000", 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.02e-06 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119096+0000", 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062507580000000004 2024-09-17T23:29:36.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119178+0000", 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2100000000000003e-05 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.119698+0000", 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00051989400000000002 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121193+0000", 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014956279999999999 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121598+0000", 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00040432800000000002 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121653+0000", 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5541999999999998e-05 2024-09-17T23:29:36.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121695+0000", 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1584e-05 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:479 3.6 3:60bdd2a4:::rbd_object_map.1152f108f2b3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.83991923, 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010595982, 2024-09-17T23:29:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 479 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112919+0000", 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3900000000000004e-07 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112923+0000", 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5379999999999998e-06 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112931+0000", 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5390000000000002e-06 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.112936+0000", 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4809999999999997e-06 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120113+0000", 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0071769390000000002 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120187+0000", 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3967000000000005e-05 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:36.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120554+0000", 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036718900000000002 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121301+0000", 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00074710800000000002 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123418+0000", 2024-09-17T23:29:36.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021164729999999998 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123475+0000", 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7598e-05 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.123515+0000", 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9310999999999997e-05 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4447.0:480 3.6 3:641496d6:::rbd_data.1152f108f2b3.0000000000000036:head [read 1069056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e23)", 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.839556996999999, 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079137459999999993, 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4447", 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/574642338", 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 480 2024-09-17T23:29:36.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113281+0000", 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113282+0000", 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3e-07 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113286+0000", 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.228e-06 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113295+0000", 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.38e-06 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.113300+0000", 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0010000000000001e-06 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.120998+0000", 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0076977790000000001 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121079+0000", 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1055000000000002e-05 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:19.121195+0000", 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115573 2024-09-17T23:29:36.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1508 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 28704768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970168+0000", 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.982669798, 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007559128, 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1508 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970168+0000", 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970168+0000", 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970169+0000", 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.63e-07 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970171+0000", 2024-09-17T23:29:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.119e-06 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970178+0000", 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0820000000000001e-06 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970191+0000", 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3671e-05 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977617+0000", 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074253929999999998 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977657+0000", 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0481000000000001e-05 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977727+0000", 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9919000000000006e-05 2024-09-17T23:29:36.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1509 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 29229056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970199+0000", 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.982638779, 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077732410000000002, 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1509 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970199+0000", 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970199+0000", 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970200+0000", 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7599999999999997e-07 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970207+0000", 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2590000000000003e-06 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970214+0000", 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5469999999999999e-06 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970220+0000", 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4820000000000002e-06 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977770+0000", 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0075506410000000003 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977861+0000", 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0414999999999994e-05 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.977972+0000", 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111421 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1510 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 29753344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.991 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.982610105999999, 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079238969999999992, 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1510 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970228+0000", 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0100000000000005e-07 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970230+0000", 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0190000000000001e-06 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970238+0000", 2024-09-17T23:29:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0979999999999994e-06 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970243+0000", 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.955e-06 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978015+0000", 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077719240000000004 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978065+0000", 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9289999999999997e-05 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978152+0000", 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7109999999999998e-05 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1511 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 30277632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970250+0000", 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.982587627000001, 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081269640000000004, 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1511 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.997 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970250+0000", 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970250+0000", 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:36.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970251+0000", 2024-09-17T23:29:36.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3000000000000001e-07 2024-09-17T23:29:36.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:36.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:36.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:36.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970253+0000", 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1289999999999999e-06 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970260+0000", 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9419999999999998e-06 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970265+0000", 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1880000000000001e-06 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978197+0000", 2024-09-17T23:29:37.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079315819999999995 2024-09-17T23:29:37.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978278+0000", 2024-09-17T23:29:37.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1754000000000004e-05 2024-09-17T23:29:37.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978377+0000", 2024-09-17T23:29:37.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8838999999999993e-05 2024-09-17T23:29:37.003 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.003 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.004 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.004 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1512 3.19 3:9eec043e:::rbd_data.11522ab9a303.0000000000000006:4 [sparse-read 30801920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:37.004 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:22.970272+0000", 2024-09-17T23:29:37.005 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.982565772999999, 2024-09-17T23:29:37.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0082278969999999996, 2024-09-17T23:29:37.005 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:37.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.006 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:37.006 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:37.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1512 2024-09-17T23:29:37.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.006 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970272+0000", 2024-09-17T23:29:37.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970272+0000", 2024-09-17T23:29:37.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970273+0000", 2024-09-17T23:29:37.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.46e-07 2024-09-17T23:29:37.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970274+0000", 2024-09-17T23:29:37.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.933e-06 2024-09-17T23:29:37.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970281+0000", 2024-09-17T23:29:37.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8290000000000002e-06 2024-09-17T23:29:37.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.970286+0000", 2024-09-17T23:29:37.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9679999999999997e-06 2024-09-17T23:29:37.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978401+0000", 2024-09-17T23:29:37.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0081144430000000007 2024-09-17T23:29:37.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978444+0000", 2024-09-17T23:29:37.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3003000000000001e-05 2024-09-17T23:29:37.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:22.978500+0000", 2024-09-17T23:29:37.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6274999999999999e-05 2024-09-17T23:29:37.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.014 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.015 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.015 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:801 3.7 3:e3b303a3:::rbd_data.1152846a29a0.0000000000000009:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:37.015 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954187+0000", 2024-09-17T23:29:37.016 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.998650445999999, 2024-09-17T23:29:37.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0080268519999999993, 2024-09-17T23:29:37.016 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 801 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954187+0000", 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954187+0000", 2024-09-17T23:29:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954188+0000", 2024-09-17T23:29:37.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8400000000000003e-07 2024-09-17T23:29:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954192+0000", 2024-09-17T23:29:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6969999999999999e-06 2024-09-17T23:29:37.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954202+0000", 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9520000000000006e-06 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954208+0000", 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2960000000000004e-06 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954268+0000", 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9689999999999999e-05 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954381+0000", 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011291899999999999 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962214+0000", 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078335139999999998 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.022 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:815 3.19 3:9aba582b:::rbd_data.1152846a29a0.0000000000000011:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954386+0000", 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.998451783, 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0079433809999999994, 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 815 2024-09-17T23:29:37.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954386+0000", 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954386+0000", 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954387+0000", 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7599999999999996e-07 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954391+0000", 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1320000000000004e-06 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954403+0000", 2024-09-17T23:29:37.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2047e-05 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954411+0000", 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4730000000000003e-06 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954490+0000", 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9579000000000005e-05 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954565+0000", 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4880000000000001e-05 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.955791+0000", 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012257220000000001 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:37.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.959032+0000", 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0032409140000000001 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962228+0000", 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031967789999999999 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962284+0000", 2024-09-17T23:29:37.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5472e-05 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962329+0000", 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5507000000000003e-05 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.029 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:816 3.1 3:85317552:::rbd_data.1152846a29a0.0000000000000012:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954432+0000", 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.998405909000001, 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0077914200000000003, 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 816 2024-09-17T23:29:37.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954432+0000", 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954432+0000", 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954433+0000", 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8100000000000003e-07 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954435+0000", 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8990000000000001e-06 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954444+0000", 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3799999999999994e-06 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954450+0000", 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9580000000000004e-06 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954738+0000", 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000287841 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954807+0000", 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.97e-05 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.955737+0000", 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000929446 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.958643+0000", 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029065810000000001 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962117+0000", 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0034735180000000001 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962190+0000", 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2727999999999999e-05 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962223+0000", 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3788000000000002e-05 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4551.0:819 3.7 3:e6fade6f:::rbd_data.1152846a29a0.0000000000000014:head [stat,delete] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:25.954498+0000", 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.998339626, 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0078496599999999996, 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4551", 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1681105503", 2024-09-17T23:29:37.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 819 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954498+0000", 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954498+0000", 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954499+0000", 2024-09-17T23:29:37.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:29:37.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954502+0000", 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3239999999999999e-06 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954511+0000", 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1870000000000003e-06 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954518+0000", 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0729999999999999e-06 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954682+0000", 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00016343600000000001 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954738+0000", 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000003e-05 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.954869+0000", 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000130402 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.956765+0000", 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018963750000000001 2024-09-17T23:29:37.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962255+0000", 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054904869999999996 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962310+0000", 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4159000000000001e-05 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:25.962348+0000", 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8362000000000003e-05 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:325 3.19 3:9df28ccf:::rbd_header.11521d8bcd2d:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.875332+0000", 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.077505657, 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010234141, 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 325 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875332+0000", 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875332+0000", 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875333+0000", 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0299999999999998e-07 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875337+0000", 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2459999999999997e-06 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875347+0000", 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0043e-05 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875353+0000", 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3859999999999998e-06 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875392+0000", 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8841000000000003e-05 2024-09-17T23:29:37.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.875396+0000", 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.07e-06 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.878416+0000", 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0030207369999999999 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.878498+0000", 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1500000000000002e-05 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.878686+0000", 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018777900000000001 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.880006+0000", 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013204429999999999 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.885504+0000", 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054983760000000001 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.885540+0000", 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5410999999999997e-05 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.885566+0000", 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6506e-05 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.050 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:438 3.8 3:123517d2:::rbd_data.11521d8bcd2d.0000000000000001:head [stat,truncate 15025664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:27.933392+0000", 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.0194458750000006, 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0083818829999999997, 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:37.051 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:37.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 438 2024-09-17T23:29:37.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.052 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933392+0000", 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933392+0000", 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933393+0000", 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.034e-06 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933399+0000", 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6509999999999998e-06 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933411+0000", 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2717e-05 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933420+0000", 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.32e-06 2024-09-17T23:29:37.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933478+0000", 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8131000000000003e-05 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for degraded object", 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.933769+0000", 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00029152699999999998 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.936587+0000", 2024-09-17T23:29:37.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028173780000000002 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.936675+0000", 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8473000000000005e-05 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.936841+0000", 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00016582200000000001 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.937289+0000", 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000448307 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.941690+0000", 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.004400798 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.941723+0000", 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2409000000000001e-05 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.941774+0000", 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1316000000000001e-05 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:2854 3.8 3:14e8b627:::rbd_header.115230678ec7:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:35.719232+0000", 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.233606102, 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0075635930000000004, 2024-09-17T23:29:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2854 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719232+0000", 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719232+0000", 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719233+0000", 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3820000000000001e-06 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719239+0000", 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:29:37.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719265+0000", 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5539000000000001e-05 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719276+0000", 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1062e-05 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.726709+0000", 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074329829999999998 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.726743+0000", 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4663000000000003e-05 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.726795+0000", 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2058999999999997e-05 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4683.0:3339 3.8 3:14e8b627:::rbd_header.115230678ec7:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:35.719672+0000", 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.2331657970000001, 2024-09-17T23:29:37.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0075848649999999997, 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4683", 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1721238999", 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719672+0000", 2024-09-17T23:29:37.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719672+0000", 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719673+0000", 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0610000000000001e-06 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719678+0000", 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.746e-06 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719687+0000", 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3200000000000006e-06 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.719694+0000", 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4790000000000001e-06 2024-09-17T23:29:37.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.727169+0000", 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074756730000000004 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.727207+0000", 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7160999999999999e-05 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.727257+0000", 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0424999999999999e-05 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4683.0:3362 3.8 3:14e8b627:::rbd_header.115230678ec7:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:35.724084+0000", 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.228754007, 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.008055207, 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4683", 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1721238999", 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3362 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.724084+0000", 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.724084+0000", 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.724084+0000", 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.15e-07 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.724089+0000", 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8300000000000003e-06 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.724096+0000", 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8419999999999999e-06 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.724100+0000", 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1640000000000003e-06 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.730940+0000", 2024-09-17T23:29:37.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.006839899 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.730985+0000", 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5268000000000001e-05 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.731109+0000", 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000123917 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:37.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.731712+0000", 2024-09-17T23:29:37.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00060246200000000001 2024-09-17T23:29:37.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:37.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.732011+0000", 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000299186 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.732041+0000", 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9785999999999999e-05 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.732139+0000", 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8437999999999995e-05 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4683.0:3367 3.8 3:14e8b627:::rbd_header.115230678ec7:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:35.725697+0000", 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.2271413449999999, 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0075314090000000002, 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:37.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4683", 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1721238999", 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3367 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725697+0000", 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725697+0000", 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725697+0000", 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5700000000000002e-07 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725700+0000", 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.117e-06 2024-09-17T23:29:37.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725708+0000", 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725712+0000", 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9879999999999998e-06 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.731878+0000", 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0061662820000000004 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.731913+0000", 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5008000000000001e-05 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.733144+0000", 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012311900000000001 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.733178+0000", 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3547000000000003e-05 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.733228+0000", 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0237e-05 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4683.0:3368 3.8 3:14e8b627:::rbd_header.115230678ec7:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:35.725866+0000", 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.2269713659999999, 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0075655590000000003, 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4683", 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1721238999", 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3368 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.081 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725866+0000", 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725866+0000", 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:37.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725867+0000", 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5600000000000005e-07 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725871+0000", 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2720000000000002e-06 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725877+0000", 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9819999999999996e-06 2024-09-17T23:29:37.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.725880+0000", 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8850000000000001e-06 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.733344+0000", 2024-09-17T23:29:37.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0074635550000000002 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.733380+0000", 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6303000000000001e-05 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:35.733432+0000", 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1805999999999998e-05 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.085 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.086 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:37.086 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:37.086 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:37.087 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone8 from 0x2723c00 (overlap 0xa10800) to 0x304e00 2024-09-17T23:29:37.088 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:37.089 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:37.085+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:37.187 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:37.186+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:37.239 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:37.240 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:37.240 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:37.240 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:37.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:37.294 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:37.292+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:37.319 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T23:29:37.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:37.366 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:37.366 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:37.366 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:37.366 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:37.367 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:37.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:37.417 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:37.414+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:37.430 INFO:teuthology.orchestra.run.smithi139.stdout:1055 write 0x51f3600 thru 0x51f37f0 (0x1f1 bytes) 2024-09-17T23:29:37.436 INFO:teuthology.orchestra.run.smithi139.stdout:1056 read 0x2438400 thru 0x2446073 (0xdc74 bytes) 2024-09-17T23:29:37.437 INFO:teuthology.orchestra.run.smithi139.stdout:1057 punch from 0xed3400 to 0xed7c00, (0x4800 bytes) 2024-09-17T23:29:37.467 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:37.467 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:37.490 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:37.489+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:37.492 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:37.498 INFO:teuthology.orchestra.run.smithi139.stdout:1059 trunc from 0x51f3800 to 0xbd75200 2024-09-17T23:29:37.510 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:37.510 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:37.532 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:37.549 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:37.549 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:37.569 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:37.570 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:37.570 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:37.570 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:37.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:37.591 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:37.589+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:37.606 INFO:teuthology.orchestra.run.smithi139.stdout:1060 write 0x32edc00 thru 0x32f58d7 (0x7cd8 bytes) 2024-09-17T23:29:37.631 INFO:teuthology.orchestra.run.smithi139.stdout:1061 write 0xdb84400 thru 0xdb86476 (0x2077 bytes) 2024-09-17T23:29:37.632 INFO:teuthology.orchestra.run.smithi139.stdout:1062 write 0x4d46800 thru 0x4d4bb34 (0x5335 bytes) 2024-09-17T23:29:37.632 INFO:teuthology.orchestra.run.smithi139.stdout:1063 read 0x40b4600 thru 0x40c3e5c (0xf85d bytes) 2024-09-17T23:29:37.645 INFO:teuthology.orchestra.run.smithi139.stdout:1067 trunc from 0xdb86600 to 0x3b8a200 2024-09-17T23:29:37.692 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:37.690+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:37.736 INFO:teuthology.orchestra.run.smithi139.stdout:1068 read 0x20a1800 thru 0x20ab5cc (0x9dcd bytes) 2024-09-17T23:29:37.739 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:37.740 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:37.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:37.778 INFO:teuthology.orchestra.run.smithi139.stdout:1069 write 0xe72a600 thru 0xe72c909 (0x230a bytes) 2024-09-17T23:29:37.779 INFO:teuthology.orchestra.run.smithi139.stdout:1071 read 0x1c91600 thru 0x1c9ff4a (0xe94b bytes) 2024-09-17T23:29:37.779 INFO:teuthology.orchestra.run.smithi139.stdout:1073 punch from 0x817a400 to 0x817c600, (0x2200 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1074 read 0xc914600 thru 0xc92442f (0xfe30 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1075 write 0xae60c00 thru 0xae64333 (0x3734 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1076 read 0x3846e00 thru 0x384e3f2 (0x75f3 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1078 read 0x6152800 thru 0x61530a4 (0x8a5 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1079 punch from 0x4e1d800 to 0x4e1e000, (0x800 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1081 punch from 0x5fd5a00 to 0x5fd6600, (0xc00 bytes) 2024-09-17T23:29:37.781 INFO:teuthology.orchestra.run.smithi139.stdout:1082 write 0x1502600 thru 0x150f9e6 (0xd3e7 bytes) 2024-09-17T23:29:37.782 INFO:teuthology.orchestra.run.smithi139.stdout:1083 write 0x3fc0600 thru 0x3fc95ea (0x8feb bytes) 2024-09-17T23:29:37.782 INFO:teuthology.orchestra.run.smithi139.stdout:1085 read 0xaceb000 thru 0xacfad31 (0xfd32 bytes) 2024-09-17T23:29:37.782 INFO:teuthology.orchestra.run.smithi139.stdout:1086 write 0x3f55200 thru 0x3f62f44 (0xdd45 bytes) 2024-09-17T23:29:37.782 INFO:teuthology.orchestra.run.smithi139.stdout:1087 write 0xdabdc00 thru 0xdabe7dd (0xbde bytes) 2024-09-17T23:29:37.793 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:37.791+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:37.796 INFO:teuthology.orchestra.run.smithi139.stdout:1089 trunc from 0xe72ca00 to 0x2c2ae00 2024-09-17T23:29:37.894 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:37.894 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:37.894 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:37.892+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:37.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:37.938 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:37.939 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:37.939 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T23:29:37.939 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.024928 2024-09-17T23:29:37.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.024928470877815356 2024-09-17T23:29:37.996 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:37.993+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:38.037 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.037 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.039 INFO:teuthology.orchestra.run.smithi139.stdout:1092 punch from 0x13d4600 to 0x13e2a00, (0xe400 bytes) 2024-09-17T23:29:38.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:38.067 INFO:teuthology.orchestra.run.smithi139.stdout:1093 punch from 0x2aad000 to 0x2ab3800, (0x6800 bytes) 2024-09-17T23:29:38.068 INFO:teuthology.orchestra.run.smithi139.stdout:1094 punch from 0xaf2c00 to 0xafa600, (0x7a00 bytes) 2024-09-17T23:29:38.097 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:38.094+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:38.198 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:38.196+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:38.200 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.201 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:38.215 INFO:teuthology.orchestra.run.smithi139.stdout:1096 trunc from 0x2c2ae00 to 0x88f2600 2024-09-17T23:29:38.299 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:38.296+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:38.311 INFO:teuthology.orchestra.run.smithi139.stdout:1098 trunc from 0x88f2600 to 0x1249400 2024-09-17T23:29:38.323 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.323 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:38.393 INFO:teuthology.orchestra.run.smithi139.stdout:1099 write 0x24a2200 thru 0x24aaf88 (0x8d89 bytes) 2024-09-17T23:29:38.400 INFO:teuthology.orchestra.run.smithi139.stdout:1100 punch from 0xc5cc00 to 0xc61400, (0x4800 bytes) 2024-09-17T23:29:38.400 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:38.398+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.498 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.500 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:38.498+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:38.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:38.578 INFO:teuthology.orchestra.run.smithi139.stdout:1101 write 0xb64d000 thru 0xb64ed80 (0x1d81 bytes) 2024-09-17T23:29:38.584 INFO:teuthology.orchestra.run.smithi139.stdout:1102 read 0x89fa800 thru 0x8a07be2 (0xd3e3 bytes) 2024-09-17T23:29:38.584 INFO:teuthology.orchestra.run.smithi139.stdout:1103 write 0x1a01400 thru 0x1a0379b (0x239c bytes) 2024-09-17T23:29:38.584 INFO:teuthology.orchestra.run.smithi139.stdout:1105 punch from 0x5487a00 to 0x548ae00, (0x3400 bytes) 2024-09-17T23:29:38.585 INFO:teuthology.orchestra.run.smithi139.stdout:1106 read 0x1326600 thru 0x132bc69 (0x566a bytes) 2024-09-17T23:29:38.586 INFO:teuthology.orchestra.run.smithi139.stdout:1107 punch from 0x61d7200 to 0x61e6000, (0xee00 bytes) 2024-09-17T23:29:38.587 INFO:teuthology.orchestra.run.smithi139.stdout:1112 write 0xe42e00 thru 0xe51f8e (0xf18f bytes) 2024-09-17T23:29:38.601 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:38.599+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:38.619 INFO:teuthology.orchestra.run.smithi139.stdout:1113 read 0x6355600 thru 0x6358cb3 (0x36b4 bytes) 2024-09-17T23:29:38.620 INFO:teuthology.orchestra.run.smithi139.stdout:1114 punch from 0xc12600 to 0xc1b400, (0x8e00 bytes) 2024-09-17T23:29:38.622 INFO:teuthology.orchestra.run.smithi139.stdout:1115 trunc from 0xb64ee00 to 0x943c200 2024-09-17T23:29:38.666 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.667 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:38.715 INFO:teuthology.orchestra.run.smithi139.stdout:1120 write 0xed15200 thru 0xed202e9 (0xb0ea bytes) 2024-09-17T23:29:38.717 INFO:teuthology.orchestra.run.smithi139.stdout:1121 clone 10 order 25 su 1048576 sc 13 2024-09-17T23:29:38.718 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:38.716+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:38.836 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:38.833+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:38.904 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:38.902+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:38.942 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:38.942 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:38.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:38.953 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 0.0249285 (816332) 2024-09-17T23:29:39.004 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:39.002+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:39.061 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:39.061 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:39.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:39.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:29:39.110 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:39.108+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:39.206 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:39.204+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:39.216 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:39.216 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:39.225 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:39.306 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:39.304+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:39.372 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:39.372 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:39.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:39.424 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:39.422+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:39.487 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:39.487 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:39.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:39.509 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:39.507+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:39.583 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:39.628 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:39.625+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:39.643 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:39.644 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:39.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:39.708 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:39.709 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:39.711 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:39.708+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:39.725 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:39.741 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:39.742 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:39.758 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:39.758 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:39.759 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:39.760 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:39.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:39.774 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:39.774 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:39.791 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:39.791 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:39.813 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:39.811+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:39.876 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:39.877 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:39.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:39.926 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:39.924+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:39.954 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:29:40.014 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:40.011+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:40.030 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone9 from 0xed20400 (overlap 0x1249400) to 0x3a8400 2024-09-17T23:29:40.041 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:40.042 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:40.042 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:40.042 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:40.055 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:40.115 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:40.113+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:40.213 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:40.213 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:40.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:40.213 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1795 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196355444999998, 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017441358000000001, 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1795 2024-09-17T23:29:40.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.04e-07 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013921+0000", 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9699999999999994e-07 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013924+0000", 2024-09-17T23:29:40.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9409999999999999e-06 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013927+0000", 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.435e-06 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031185+0000", 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017258909999999999 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031263+0000", 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7015999999999996e-05 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031361+0000", 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8855000000000001e-05 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.218 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1796 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196345011999998, 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017564354000000001, 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1796 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.219 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013931+0000", 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-07 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013935+0000", 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5859999999999998e-06 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013939+0000", 2024-09-17T23:29:40.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.473e-06 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013941+0000", 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.492e-06 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031391+0000", 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017449416999999998 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031438+0000", 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6885999999999998e-05 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031495+0000", 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7281999999999998e-05 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1797 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 9830400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:40.223 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196330680999999, 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017763582999999999, 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1797 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2600000000000001e-07 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013946+0000", 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.071e-06 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013949+0000", 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1020000000000001e-06 2024-09-17T23:29:40.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013952+0000", 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1039999999999998e-06 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031533+0000", 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017580312000000001 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031612+0000", 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9287000000000002e-05 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031708+0000", 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6481e-05 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1798 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196319977000002, 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017889677, 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1798 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013956+0000", 2024-09-17T23:29:40.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3900000000000001e-07 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013957+0000", 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.093e-06 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013960+0000", 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0989999999999999e-06 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013969+0000", 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.67e-06 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031737+0000", 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017768055000000001 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031784+0000", 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7057e-05 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031845+0000", 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1464000000000006e-05 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1799 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196291626000001, 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018073763999999999, 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1799 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0900000000000002e-07 2024-09-17T23:29:40.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013987+0000", 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.379e-06 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013994+0000", 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4429999999999997e-06 2024-09-17T23:29:40.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014000+0000", 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3469999999999998e-06 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031883+0000", 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017883546 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031962+0000", 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9000999999999998e-05 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032058+0000", 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5538999999999994e-05 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1800 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196266282, 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018179992999999998, 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1800 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014010+0000", 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8299999999999997e-07 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014012+0000", 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1160000000000002e-06 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014020+0000", 2024-09-17T23:29:40.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3340000000000002e-06 2024-09-17T23:29:40.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014025+0000", 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2429999999999996e-06 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032086+0000", 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018060579 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032132+0000", 2024-09-17T23:29:40.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6464000000000001e-05 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032189+0000", 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6774000000000002e-05 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1801 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196241883999999, 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018361467999999999, 2024-09-17T23:29:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1801 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999999999999999e-07 2024-09-17T23:29:40.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014036+0000", 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014044+0000", 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1829999999999994e-06 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014051+0000", 2024-09-17T23:29:40.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9260000000000003e-06 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032228+0000", 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018176906 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032304+0000", 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939000000000006e-05 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032395+0000", 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0845000000000002e-05 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1803 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.251 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196214616999999, 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018464566000000002, 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1803 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014063+0000", 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0779999999999998e-06 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014078+0000", 2024-09-17T23:29:40.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4413000000000001e-05 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014084+0000", 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032429+0000", 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018344972000000001 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032469+0000", 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0179000000000003e-05 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032525+0000", 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6561999999999998e-05 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.256 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1805 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.19617255, 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018638358000000001, 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.257 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1805 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:40.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4099999999999999e-07 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014106+0000", 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6460000000000002e-06 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014116+0000", 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8689999999999999e-06 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014124+0000", 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2139999999999996e-06 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032564+0000", 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018440070999999999 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032639+0000", 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4868000000000002e-05 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032741+0000", 2024-09-17T23:29:40.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102149 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1810 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196141227999998, 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018741180999999999, 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1810 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014135+0000", 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014144+0000", 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0629999999999995e-06 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014151+0000", 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5059999999999998e-06 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014163+0000", 2024-09-17T23:29:40.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1399e-05 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032774+0000", 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018610924000000001 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032818+0000", 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3927999999999997e-05 2024-09-17T23:29:40.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032875+0000", 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7896000000000003e-05 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1814 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:40.267 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196103827000002, 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018916773000000001, 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1814 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:40.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6300000000000005e-07 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014175+0000", 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4739999999999999e-06 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014182+0000", 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0990000000000001e-06 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014187+0000", 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2630000000000003e-06 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032915+0000", 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018727573000000001 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032990+0000", 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5766000000000006e-05 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033088+0000", 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7935000000000003e-05 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1822 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196079421, 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019019635999999999, 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1822 2024-09-17T23:29:40.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6199999999999998e-07 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014199+0000", 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3309999999999998e-06 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014206+0000", 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.148e-06 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014212+0000", 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9490000000000001e-06 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033118+0000", 2024-09-17T23:29:40.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018906164 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033159+0000", 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1230999999999999e-05 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033216+0000", 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6351000000000003e-05 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.277 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1826 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196048851, 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019198232999999999, 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1826 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014229+0000", 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8419999999999999e-06 2024-09-17T23:29:40.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014233+0000", 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1150000000000004e-06 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014242+0000", 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1900000000000001e-06 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033254+0000", 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019012228999999999 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033328+0000", 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3709999999999997e-05 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033425+0000", 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6754000000000002e-05 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1830 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196028635000001, 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019304154, 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1830 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.15e-07 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014252+0000", 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0939999999999998e-06 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014257+0000", 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3610000000000003e-06 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014259+0000", 2024-09-17T23:29:40.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6350000000000002e-06 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033453+0000", 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019193661000000001 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033495+0000", 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:29:40.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033551+0000", 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168000000000002e-05 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1834 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:40.288 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.196012290999999, 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019503455999999999, 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1834 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:40.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4499999999999998e-07 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014265+0000", 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1400000000000001e-06 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014269+0000", 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6149999999999999e-06 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014272+0000", 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5220000000000002e-06 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033591+0000", 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019318887 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033668+0000", 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7348000000000006e-05 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033767+0000", 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8698999999999998e-05 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1837 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.195999598, 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019625881000000001, 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1837 2024-09-17T23:29:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.28e-07 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014277+0000", 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.145e-06 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014280+0000", 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1130000000000001e-06 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014283+0000", 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3369999999999998e-06 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033798+0000", 2024-09-17T23:29:40.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019515498999999999 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033844+0000", 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5553000000000002e-05 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033902+0000", 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8006e-05 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.298 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1839 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.195989088000001, 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019823990999999999, 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1839 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014287+0000", 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014288+0000", 2024-09-17T23:29:40.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.133e-06 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014291+0000", 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0479999999999999e-06 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014294+0000", 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9160000000000001e-06 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033941+0000", 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019646961000000001 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034015+0000", 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4411000000000003e-05 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034110+0000", 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5313e-05 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1841 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.195978406999998, 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019940504000000001, 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1841 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:40.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2999999999999999e-07 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014298+0000", 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0249999999999999e-06 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014302+0000", 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014304+0000", 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3209999999999999e-06 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034143+0000", 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019838880999999999 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034183+0000", 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9988999999999999e-05 2024-09-17T23:29:40.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034238+0000", 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4833999999999997e-05 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:14772 3.12 3:4a278708:::rbd_data.1152c1142907.000000000000002f:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.309 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:40.034843+0000", 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.17543228299999999, 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018280547000000001, 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 14772 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034843+0000", 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034843+0000", 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034844+0000", 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3499999999999996e-07 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034847+0000", 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6480000000000001e-06 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034855+0000", 2024-09-17T23:29:40.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9670000000000001e-06 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034862+0000", 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9450000000000004e-06 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034911+0000", 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8934999999999998e-05 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034978+0000", 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7001000000000004e-05 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035135+0000", 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000157176 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.038818+0000", 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0036822170000000002 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.053005+0000", 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014187297 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.053070+0000", 2024-09-17T23:29:40.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4719000000000002e-05 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.053124+0000", 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4007e-05 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.316 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:14823 3.12 3:48da8422:::rbd_data.1152c1142907.0000000000000049:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:40.035701+0000", 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.17457410100000001, 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019721108000000001, 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 14823 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.317 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035701+0000", 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035701+0000", 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035702+0000", 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0399999999999996e-07 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035705+0000", 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6929999999999999e-06 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:40.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035711+0000", 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7630000000000003e-06 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035716+0000", 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8219999999999997e-06 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.037718+0000", 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020017839999999999 2024-09-17T23:29:40.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.037789+0000", 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1569999999999994e-05 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.037966+0000", 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000176639 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.039484+0000", 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001518078 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.055324+0000", 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.015839539999999999 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.055382+0000", 2024-09-17T23:29:40.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8349e-05 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.055422+0000", 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0365999999999999e-05 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:40.323 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:40.325 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:40.322+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:40.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:40.426 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:40.424+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:40.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22407 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.460606+0000", 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012928868, 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012952012000000001, 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22407 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460606+0000", 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460606+0000", 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460606+0000", 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-07 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460612+0000", 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8619999999999999e-06 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460621+0000", 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2359999999999993e-06 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460627+0000", 2024-09-17T23:29:40.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22714 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461585+0000", 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01195007, 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204595, 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22714 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461585+0000", 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461585+0000", 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461585+0000", 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461587+0000", 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461594+0000", 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1470000000000004e-06 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461602+0000", 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1659999999999997e-06 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22746 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462467+0000", 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011067696, 2024-09-17T23:29:40.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011224958, 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22746 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462467+0000", 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462467+0000", 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462468+0000", 2024-09-17T23:29:40.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462470+0000", 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.154e-06 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462476+0000", 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.516e-06 2024-09-17T23:29:40.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462483+0000", 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6810000000000001e-06 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22778 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463079+0000", 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010455496, 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010672816, 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22778 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.504 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463079+0000", 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463079+0000", 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463079+0000", 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463081+0000", 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.147e-06 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463085+0000", 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0770000000000001e-06 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463087+0000", 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22980 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463735+0000", 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097994620000000001, 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010074207999999999, 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22980 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463735+0000", 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463735+0000", 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463735+0000", 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.99e-07 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463736+0000", 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0410000000000001e-06 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463739+0000", 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9950000000000001e-06 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463742+0000", 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23012 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464403+0000", 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091313209999999995, 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094661569999999993, 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23012 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464403+0000", 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464403+0000", 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464404+0000", 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-07 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464405+0000", 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.094e-06 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464408+0000", 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7629999999999998e-06 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464411+0000", 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6240000000000002e-06 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23215 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465080+0000", 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084544960000000006, 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088420219999999997, 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23215 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465080+0000", 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465080+0000", 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465080+0000", 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4499999999999998e-07 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465082+0000", 2024-09-17T23:29:40.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1349999999999999e-06 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465085+0000", 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0989999999999999e-06 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465087+0000", 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7039999999999999e-06 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23247 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.469202+0000", 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0043324510000000002, 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047749680000000001, 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23247 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469202+0000", 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469202+0000", 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469308+0000", 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106253 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469312+0000", 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7400000000000002e-06 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469321+0000", 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2830000000000001e-06 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469327+0000", 2024-09-17T23:29:40.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5209999999999999e-06 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22408 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.460635+0000", 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012900027999999999, 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01340068, 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22408 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460635+0000", 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460635+0000", 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460635+0000", 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460638+0000", 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3599999999999999e-06 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460643+0000", 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8429999999999998e-06 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460648+0000", 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.78e-06 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22715 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461615+0000", 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011919509, 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012482854999999999, 2024-09-17T23:29:40.528 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22715 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461615+0000", 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461615+0000", 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461615+0000", 2024-09-17T23:29:40.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0800000000000001e-07 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461617+0000", 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9410000000000001e-06 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461621+0000", 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8650000000000003e-06 2024-09-17T23:29:40.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461624+0000", 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22747 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462490+0000", 2024-09-17T23:29:40.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011044874, 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011668540999999999, 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22747 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462490+0000", 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462490+0000", 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462490+0000", 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462492+0000", 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462499+0000", 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3960000000000003e-06 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462504+0000", 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-06 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22779 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463090+0000", 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010444386999999999, 2024-09-17T23:29:40.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011124853000000001, 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22779 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463090+0000", 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463090+0000", 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463091+0000", 2024-09-17T23:29:40.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463092+0000", 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2240000000000001e-06 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463096+0000", 2024-09-17T23:29:40.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2309999999999998e-06 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463103+0000", 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9940000000000003e-06 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22981 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.540 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463745+0000", 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097897399999999999, 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010524652000000001, 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22981 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463745+0000", 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463745+0000", 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463745+0000", 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-07 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463746+0000", 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1799999999999999e-06 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463749+0000", 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9849999999999998e-06 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463752+0000", 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4030000000000001e-06 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23013 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464414+0000", 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091205460000000002, 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0099086039999999997, 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23013 2024-09-17T23:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464414+0000", 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464414+0000", 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464414+0000", 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1400000000000001e-07 2024-09-17T23:29:40.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464416+0000", 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464419+0000", 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464421+0000", 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4679999999999999e-06 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23216 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465092+0000", 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084428409999999995, 2024-09-17T23:29:40.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092843140000000001, 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23216 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465092+0000", 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465092+0000", 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465092+0000", 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:29:40.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465093+0000", 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.387e-06 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465097+0000", 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-17T23:29:40.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465100+0000", 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.447e-06 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23248 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.469340+0000", 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0041947570000000003, 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050946560000000004, 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23248 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469340+0000", 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469340+0000", 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469473+0000", 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013328099999999999 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469476+0000", 2024-09-17T23:29:40.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1879999999999998e-06 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469487+0000", 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0159e-05 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469491+0000", 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0249999999999996e-06 2024-09-17T23:29:40.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22409 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.460669+0000", 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012865602, 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013823012000000001, 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22409 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460669+0000", 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460669+0000", 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460669+0000", 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460671+0000", 2024-09-17T23:29:40.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7719999999999999e-06 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460675+0000", 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7720000000000001e-06 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.460678+0000", 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-17T23:29:40.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22716 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461637+0000", 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011897329999999999, 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012916741000000001, 2024-09-17T23:29:40.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22716 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461637+0000", 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461637+0000", 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461638+0000", 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-07 2024-09-17T23:29:40.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461640+0000", 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.142e-06 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461647+0000", 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.109e-06 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461654+0000", 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5579999999999997e-06 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22748 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462515+0000", 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011020135, 2024-09-17T23:29:40.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012102961000000001, 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22748 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462515+0000", 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462515+0000", 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462515+0000", 2024-09-17T23:29:40.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7000000000000004e-07 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462518+0000", 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462525+0000", 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:29:40.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462532+0000", 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0319999999999998e-06 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22780 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463123+0000", 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010411576000000001, 2024-09-17T23:29:40.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011553145000000001, 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22780 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463123+0000", 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463123+0000", 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463124+0000", 2024-09-17T23:29:40.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463126+0000", 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.294e-06 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463134+0000", 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8089999999999993e-06 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463140+0000", 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4570000000000002e-06 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22982 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463755+0000", 2024-09-17T23:29:40.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097799429999999993, 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010978778999999999, 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22982 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463755+0000", 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463755+0000", 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463755+0000", 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.23e-07 2024-09-17T23:29:40.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463756+0000", 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0669999999999999e-06 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463766+0000", 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0448999999999999e-05 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463772+0000", 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5609999999999998e-06 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23014 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.578 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464424+0000", 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091106539999999993, 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010367566999999999, 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23014 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.579 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464424+0000", 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464424+0000", 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464424+0000", 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.29e-07 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464425+0000", 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0499999999999999e-06 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464428+0000", 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.022e-06 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464433+0000", 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4560000000000002e-06 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23217 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465103+0000", 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084321139999999992, 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097466549999999999, 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23217 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465103+0000", 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465103+0000", 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465103+0000", 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465104+0000", 2024-09-17T23:29:40.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.11e-06 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465107+0000", 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0929999999999999e-06 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465110+0000", 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5469999999999999e-06 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23249 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.469498+0000", 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0040366559999999996, 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054068249999999997, 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23249 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469498+0000", 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469498+0000", 2024-09-17T23:29:40.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469583+0000", 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4753999999999995e-05 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469591+0000", 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0439999999999996e-06 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469605+0000", 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4205e-05 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469614+0000", 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1910000000000006e-06 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22717 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461661+0000", 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011874091999999999, 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013301284999999999, 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22717 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461661+0000", 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461661+0000", 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461661+0000", 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-07 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461663+0000", 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:29:40.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461669+0000", 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9059999999999999e-06 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461673+0000", 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.403e-06 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22749 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462539+0000", 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010995385999999999, 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012476589999999999, 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22749 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462539+0000", 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462539+0000", 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462540+0000", 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462542+0000", 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462548+0000", 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0859999999999998e-06 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462553+0000", 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9180000000000002e-06 2024-09-17T23:29:40.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22781 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463148+0000", 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010387014, 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011923714, 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22781 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463148+0000", 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463148+0000", 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463148+0000", 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463150+0000", 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1689999999999999e-06 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463156+0000", 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.161e-06 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463164+0000", 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-09-17T23:29:40.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22983 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463786+0000", 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097483919999999998, 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011340329999999999, 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22983 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463786+0000", 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463786+0000", 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463787+0000", 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463793+0000", 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463805+0000", 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1918e-05 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463812+0000", 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0990000000000001e-06 2024-09-17T23:29:40.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23015 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464438+0000", 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0090962930000000001, 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010742903, 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23015 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464438+0000", 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464438+0000", 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464439+0000", 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7000000000000004e-07 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464441+0000", 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.863e-06 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464447+0000", 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0730000000000001e-06 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464452+0000", 2024-09-17T23:29:40.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23218 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465114+0000", 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084208760000000007, 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010121887, 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23218 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465114+0000", 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465114+0000", 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465114+0000", 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1299999999999999e-07 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465115+0000", 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.167e-06 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465118+0000", 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0290000000000001e-06 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465121+0000", 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5629999999999999e-06 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23250 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.469628+0000", 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0039067629999999997, 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056627730000000003, 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23250 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469628+0000", 2024-09-17T23:29:40.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469628+0000", 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469800+0000", 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017223999999999999 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469804+0000", 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3670000000000001e-06 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469810+0000", 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1449999999999996e-06 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469814+0000", 2024-09-17T23:29:40.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.929e-06 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22718 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461695+0000", 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011839206999999999, 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013652014000000001, 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22718 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461695+0000", 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461695+0000", 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461696+0000", 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.65e-07 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461702+0000", 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7089999999999999e-06 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461706+0000", 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9289999999999997e-06 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461712+0000", 2024-09-17T23:29:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2789999999999998e-06 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22750 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462559+0000", 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010975245, 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012844028, 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22750 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462559+0000", 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462559+0000", 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462564+0000", 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6380000000000003e-06 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462566+0000", 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2900000000000001e-06 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462573+0000", 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462579+0000", 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1480000000000003e-06 2024-09-17T23:29:40.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22782 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463171+0000", 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010363537000000001, 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012288633, 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22782 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463171+0000", 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463171+0000", 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463172+0000", 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463174+0000", 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4109999999999999e-06 2024-09-17T23:29:40.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463182+0000", 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3960000000000006e-06 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463188+0000", 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1769999999999997e-06 2024-09-17T23:29:40.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22984 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463821+0000", 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097140330000000004, 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011703346, 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22984 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463821+0000", 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463821+0000", 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463821+0000", 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463824+0000", 2024-09-17T23:29:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463831+0000", 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463841+0000", 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:29:40.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23016 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464460+0000", 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.009075006, 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011142040000000001, 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23016 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464460+0000", 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464460+0000", 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464460+0000", 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464462+0000", 2024-09-17T23:29:40.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.108e-06 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464474+0000", 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1437000000000001e-05 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464480+0000", 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4330000000000001e-06 2024-09-17T23:29:40.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23219 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465124+0000", 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084107910000000008, 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010535882999999999, 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23219 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465124+0000", 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465124+0000", 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465124+0000", 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.29e-07 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465125+0000", 2024-09-17T23:29:40.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0559999999999999e-06 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465129+0000", 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999998e-06 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465135+0000", 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23251 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.469824+0000", 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0037106119999999999, 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005890246, 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23251 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469824+0000", 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469824+0000", 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469925+0000", 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010107000000000001 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469929+0000", 2024-09-17T23:29:40.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.659e-06 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469940+0000", 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0967e-05 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469947+0000", 2024-09-17T23:29:40.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5020000000000003e-06 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22719 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461717+0000", 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011817253999999999, 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014054054999999999, 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22719 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461717+0000", 2024-09-17T23:29:40.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461717+0000", 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461718+0000", 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1099999999999996e-07 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461720+0000", 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461726+0000", 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461731+0000", 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22751 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462586+0000", 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010948294000000001, 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013238345, 2024-09-17T23:29:40.651 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22751 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462586+0000", 2024-09-17T23:29:40.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462586+0000", 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462587+0000", 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462589+0000", 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1789999999999998e-06 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462596+0000", 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.533e-06 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462600+0000", 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7570000000000004e-06 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22783 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463202+0000", 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010333042000000001, 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012678853, 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22783 2024-09-17T23:29:40.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463202+0000", 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463202+0000", 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463202+0000", 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463204+0000", 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2579999999999998e-06 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463212+0000", 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9680000000000006e-06 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463217+0000", 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1189999999999996e-06 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22985 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463848+0000", 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096866450000000007, 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012089153, 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22985 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463848+0000", 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463848+0000", 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463848+0000", 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463851+0000", 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2299999999999998e-06 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463858+0000", 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2110000000000001e-06 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463863+0000", 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.028e-06 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23017 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464489+0000", 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0090461819999999998, 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011502501, 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23017 2024-09-17T23:29:40.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464489+0000", 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464489+0000", 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464489+0000", 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464491+0000", 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2390000000000001e-06 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464498+0000", 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5230000000000001e-06 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464504+0000", 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1079999999999998e-06 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23220 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465154+0000", 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083810010000000008, 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010892324, 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23220 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465154+0000", 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465154+0000", 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465154+0000", 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4899999999999995e-07 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465157+0000", 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465163+0000", 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465168+0000", 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23252 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.469960+0000", 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0035751450000000001, 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061406509999999996, 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23252 2024-09-17T23:29:40.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469960+0000", 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.469960+0000", 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470240+0000", 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028087400000000002 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470244+0000", 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3390000000000001e-06 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470250+0000", 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1340000000000001e-06 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470257+0000", 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2760000000000003e-06 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:3911 3.9 3:929ef850:::rbd_header.1152c1142907:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.475404+0000", 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9981303, 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035900199999999998, 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475404+0000", 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475404+0000", 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475405+0000", 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6599999999999995e-07 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475410+0000", 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475419+0000", 2024-09-17T23:29:40.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0210000000000005e-06 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475423+0000", 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.172e-06 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475670+0000", 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024678200000000002 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475716+0000", 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6554999999999997e-05 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.475763+0000", 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7135000000000002e-05 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22720 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461737+0000", 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011797406999999999, 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014494709999999999, 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22720 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461737+0000", 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461737+0000", 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461738+0000", 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461740+0000", 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9989999999999998e-06 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461746+0000", 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0909999999999998e-06 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461751+0000", 2024-09-17T23:29:40.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.904e-06 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22752 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462613+0000", 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010921548999999999, 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01367726, 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22752 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462613+0000", 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462613+0000", 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462614+0000", 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462616+0000", 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462622+0000", 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2160000000000003e-06 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462627+0000", 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5950000000000001e-06 2024-09-17T23:29:40.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22954 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463224+0000", 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010310227999999999, 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013119408000000001, 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22954 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463224+0000", 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463224+0000", 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463225+0000", 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7e-07 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463227+0000", 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1789999999999998e-06 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463233+0000", 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2910000000000004e-06 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463238+0000", 2024-09-17T23:29:40.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-06 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22986 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463869+0000", 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096654870000000004, 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012529751, 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22986 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463869+0000", 2024-09-17T23:29:40.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463869+0000", 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463870+0000", 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2899999999999999e-07 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463872+0000", 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9120000000000001e-06 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463879+0000", 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3030000000000002e-06 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463884+0000", 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8049999999999997e-06 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23018 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464511+0000", 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0090238550000000008, 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011945102000000001, 2024-09-17T23:29:40.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23018 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464511+0000", 2024-09-17T23:29:40.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464511+0000", 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464511+0000", 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:29:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464513+0000", 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9769999999999999e-06 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464522+0000", 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4530000000000002e-06 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464527+0000", 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23221 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465176+0000", 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083590980000000006, 2024-09-17T23:29:40.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011335088, 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23221 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465176+0000", 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465176+0000", 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465176+0000", 2024-09-17T23:29:40.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9700000000000002e-07 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465178+0000", 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.013e-06 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465189+0000", 2024-09-17T23:29:40.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1273e-05 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465197+0000", 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4080000000000001e-06 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23253 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.470283+0000", 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0032514279999999998, 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062892139999999996, 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23253 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470283+0000", 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470283+0000", 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470284+0000", 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470287+0000", 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8229999999999999e-06 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470295+0000", 2024-09-17T23:29:40.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1869999999999997e-06 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470302+0000", 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8050000000000001e-06 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.709 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22721 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461758+0000", 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011777017000000001, 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014870081, 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22721 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461758+0000", 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461758+0000", 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461758+0000", 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461760+0000", 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9860000000000001e-06 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461764+0000", 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8940000000000003e-06 2024-09-17T23:29:40.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461767+0000", 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.807e-06 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22753 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462633+0000", 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010901737999999999, 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014050194, 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22753 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462633+0000", 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462633+0000", 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462633+0000", 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462639+0000", 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7030000000000003e-06 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462645+0000", 2024-09-17T23:29:40.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462649+0000", 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4270000000000001e-06 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22955 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463245+0000", 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010290153, 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013495261999999999, 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22955 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463245+0000", 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463245+0000", 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463245+0000", 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9699999999999996e-07 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463247+0000", 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0810000000000001e-06 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463256+0000", 2024-09-17T23:29:40.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2150000000000007e-06 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463264+0000", 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529999999999992e-06 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22987 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463892+0000", 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096429640000000004, 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012902767000000001, 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22987 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463892+0000", 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463892+0000", 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463892+0000", 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463894+0000", 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1490000000000001e-06 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463900+0000", 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1959999999999996e-06 2024-09-17T23:29:40.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463905+0000", 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8849999999999998e-06 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23019 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464540+0000", 2024-09-17T23:29:40.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0089947240000000008, 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012308069, 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23019 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464540+0000", 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464540+0000", 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464541+0000", 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4199999999999996e-07 2024-09-17T23:29:40.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464543+0000", 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5179999999999999e-06 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464550+0000", 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7460000000000003e-06 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464556+0000", 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3219999999999998e-06 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23222 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465206+0000", 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083288359999999992, 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011695478, 2024-09-17T23:29:40.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23222 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465206+0000", 2024-09-17T23:29:40.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465206+0000", 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465206+0000", 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465209+0000", 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4609999999999998e-06 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465216+0000", 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6369999999999999e-06 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465223+0000", 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6719999999999998e-06 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23254 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.470310+0000", 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0032247109999999999, 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066449839999999996, 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23254 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470310+0000", 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470310+0000", 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470567+0000", 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000256848 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470570+0000", 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0809999999999998e-06 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470577+0000", 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470581+0000", 2024-09-17T23:29:40.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1760000000000003e-06 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22722 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461779+0000", 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011755211, 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015241485000000001, 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22722 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461779+0000", 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461779+0000", 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461780+0000", 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8599999999999999e-07 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461781+0000", 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6190000000000001e-06 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461785+0000", 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461788+0000", 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-09-17T23:29:40.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22754 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462655+0000", 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010879524, 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014423735, 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22754 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462655+0000", 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462655+0000", 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462656+0000", 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462657+0000", 2024-09-17T23:29:40.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.694e-06 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462666+0000", 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3299999999999999e-06 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462670+0000", 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4719999999999997e-06 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22956 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463278+0000", 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010256688999999999, 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013856637, 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22956 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463278+0000", 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463278+0000", 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463278+0000", 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-07 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463281+0000", 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4399999999999999e-06 2024-09-17T23:29:40.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463290+0000", 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7940000000000008e-06 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463297+0000", 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2609999999999996e-06 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22988 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463921+0000", 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096138390000000008, 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013267678, 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22988 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463921+0000", 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463921+0000", 2024-09-17T23:29:40.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463921+0000", 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463924+0000", 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.339e-06 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463930+0000", 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5610000000000004e-06 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463935+0000", 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1329999999999998e-06 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23020 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464571+0000", 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0089634640000000008, 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012674359, 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23020 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464571+0000", 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464571+0000", 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464572+0000", 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7e-07 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464574+0000", 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464581+0000", 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464587+0000", 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0680000000000002e-06 2024-09-17T23:29:40.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23223 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465232+0000", 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083024640000000007, 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012066973999999999, 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23223 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465232+0000", 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465232+0000", 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465233+0000", 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465235+0000", 2024-09-17T23:29:40.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465244+0000", 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1460000000000002e-06 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465251+0000", 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23255 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.470597+0000", 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0029372259999999998, 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067598739999999999, 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23255 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470597+0000", 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.470597+0000", 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472567+0000", 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019694640000000002 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472571+0000", 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5260000000000004e-06 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472581+0000", 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4760000000000004e-06 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472596+0000", 2024-09-17T23:29:40.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4704e-05 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22723 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461792+0000", 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011742275, 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015620382, 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22723 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461792+0000", 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461792+0000", 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461793+0000", 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461794+0000", 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.128e-06 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461799+0000", 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4809999999999997e-06 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461802+0000", 2024-09-17T23:29:40.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.224e-06 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22755 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462675+0000", 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010859835999999999, 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014818191, 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22755 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462675+0000", 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462675+0000", 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462675+0000", 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.75e-07 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462677+0000", 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.641e-06 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462683+0000", 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2020000000000001e-06 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462688+0000", 2024-09-17T23:29:40.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3409999999999996e-06 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22957 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463304+0000", 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010230380000000001, 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014242530999999999, 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22957 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463304+0000", 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463304+0000", 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463305+0000", 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463307+0000", 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.328e-06 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463315+0000", 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7589999999999997e-06 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463321+0000", 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5860000000000001e-06 2024-09-17T23:29:40.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22989 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463944+0000", 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.009590899, 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013665025000000001, 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22989 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463944+0000", 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463944+0000", 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463944+0000", 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463946+0000", 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.159e-06 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463953+0000", 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463961+0000", 2024-09-17T23:29:40.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1419999999999993e-06 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23021 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464596+0000", 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0089389359999999998, 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013067618, 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23021 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464596+0000", 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464596+0000", 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464596+0000", 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6100000000000002e-07 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464598+0000", 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0789999999999999e-06 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464605+0000", 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464610+0000", 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9180000000000002e-06 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23224 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465259+0000", 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082759170000000007, 2024-09-17T23:29:40.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012462891, 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23224 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465259+0000", 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465259+0000", 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465259+0000", 2024-09-17T23:29:40.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465261+0000", 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2220000000000001e-06 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465268+0000", 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5769999999999999e-06 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465273+0000", 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8489999999999997e-06 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23256 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472607+0000", 2024-09-17T23:29:40.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00092730999999999998, 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051675209999999996, 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23256 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472607+0000", 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472607+0000", 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472608+0000", 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0999999999999999e-07 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472611+0000", 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.886e-06 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472618+0000", 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9020000000000002e-06 2024-09-17T23:29:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472623+0000", 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.728e-06 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22724 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461844+0000", 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011690258, 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015985514999999999, 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22724 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461844+0000", 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461844+0000", 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461845+0000", 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0199999999999998e-07 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461847+0000", 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8530000000000001e-06 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461852+0000", 2024-09-17T23:29:40.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7209999999999999e-06 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461857+0000", 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7879999999999997e-06 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22756 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462692+0000", 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010842421, 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015191219000000001, 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22756 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462692+0000", 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462692+0000", 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462693+0000", 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0100000000000001e-07 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462694+0000", 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.609e-06 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462700+0000", 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9020000000000004e-06 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462708+0000", 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.072e-06 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22958 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463330+0000", 2024-09-17T23:29:40.801 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010204509, 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014614696, 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22958 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463330+0000", 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463330+0000", 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463331+0000", 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.51e-07 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463333+0000", 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3499999999999999e-06 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463342+0000", 2024-09-17T23:29:40.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3700000000000001e-06 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463348+0000", 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3530000000000002e-06 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.805 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22990 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463968+0000", 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095666680000000004, 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014031132999999999, 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22990 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.806 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463968+0000", 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463968+0000", 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463968+0000", 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463971+0000", 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3439999999999999e-06 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463979+0000", 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463985+0000", 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2759999999999997e-06 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23022 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464623+0000", 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0089115759999999992, 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013448949, 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23022 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464623+0000", 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464623+0000", 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464624+0000", 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464626+0000", 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3429999999999998e-06 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464634+0000", 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9049999999999997e-06 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464648+0000", 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4201e-05 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23225 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465279+0000", 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082554889999999995, 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012846801999999999, 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23225 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465279+0000", 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465279+0000", 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465409+0000", 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129849 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465415+0000", 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9950000000000002e-06 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465422+0000", 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465426+0000", 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1069999999999998e-06 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23257 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472631+0000", 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00090384500000000002, 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055490060000000004, 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23257 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472631+0000", 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472631+0000", 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472631+0000", 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472634+0000", 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472644+0000", 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472649+0000", 2024-09-17T23:29:40.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6060000000000002e-06 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:3924 3.9 3.a1f7949 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.477656+0000", 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9958787, 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058054999999999999, 2024-09-17T23:29:40.822 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3924 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477656+0000", 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477656+0000", 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477657+0000", 2024-09-17T23:29:40.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477660+0000", 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0529999999999998e-06 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477665+0000", 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7740000000000001e-06 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477669+0000", 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7330000000000001e-06 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22725 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.461874+0000", 2024-09-17T23:29:40.826 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011661169000000001, 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016416554999999999, 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22725 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461874+0000", 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461874+0000", 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461874+0000", 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461876+0000", 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1220000000000002e-06 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461886+0000", 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9049999999999992e-06 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.461892+0000", 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6289999999999998e-06 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22757 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462714+0000", 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010820465, 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015629503999999999, 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22757 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462714+0000", 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462714+0000", 2024-09-17T23:29:40.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462715+0000", 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462716+0000", 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6190000000000001e-06 2024-09-17T23:29:40.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462722+0000", 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1179999999999997e-06 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462727+0000", 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4390000000000002e-06 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22959 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463354+0000", 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010180407000000001, 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015043798000000001, 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22959 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463354+0000", 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463354+0000", 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463355+0000", 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463361+0000", 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463365+0000", 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9230000000000004e-06 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463368+0000", 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9160000000000001e-06 2024-09-17T23:29:40.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22991 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464007+0000", 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095275859999999993, 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014448404, 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22991 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464007+0000", 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464007+0000", 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464007+0000", 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.65e-07 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464010+0000", 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1409999999999999e-06 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464014+0000", 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7509999999999999e-06 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464017+0000", 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0350000000000002e-06 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23023 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464661+0000", 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0088739879999999993, 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013849185999999999, 2024-09-17T23:29:40.843 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23023 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464661+0000", 2024-09-17T23:29:40.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464661+0000", 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464661+0000", 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-17T23:29:40.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464664+0000", 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.441e-06 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464672+0000", 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1839999999999999e-06 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464678+0000", 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7379999999999999e-06 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23226 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.465435+0000", 2024-09-17T23:29:40.847 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0080999660000000001, 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013137845, 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23226 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465435+0000", 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.465435+0000", 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466567+0000", 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011319119999999999 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466571+0000", 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6630000000000001e-06 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466582+0000", 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466589+0000", 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6619999999999999e-06 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23258 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472662+0000", 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00087230499999999998, 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059638319999999996, 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23258 2024-09-17T23:29:40.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472662+0000", 2024-09-17T23:29:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472662+0000", 2024-09-17T23:29:40.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472663+0000", 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472665+0000", 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5229999999999998e-06 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472673+0000", 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472680+0000", 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.64e-06 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:3925 3.9 3.a1f7949 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.477793+0000", 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9957414, 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00088672499999999999, 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:29:40.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3925 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477793+0000", 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477793+0000", 2024-09-17T23:29:40.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477794+0000", 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477796+0000", 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.576e-06 2024-09-17T23:29:40.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477801+0000", 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8670000000000002e-06 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477805+0000", 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3440000000000001e-06 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22758 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.462730+0000", 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010804655, 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016006609000000002, 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22758 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462730+0000", 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462730+0000", 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462730+0000", 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2399999999999999e-07 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462731+0000", 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1260000000000001e-06 2024-09-17T23:29:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462734+0000", 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0280000000000001e-06 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.462737+0000", 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4099999999999998e-06 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22960 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463372+0000", 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010162440999999999, 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015417779, 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22960 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463372+0000", 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463372+0000", 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463372+0000", 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4299999999999999e-07 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463374+0000", 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:29:40.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463377+0000", 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1860000000000001e-06 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463380+0000", 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5469999999999999e-06 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22992 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464022+0000", 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095130779999999995, 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014820045, 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22992 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464022+0000", 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464022+0000", 2024-09-17T23:29:40.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464022+0000", 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3999999999999997e-07 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464023+0000", 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2780000000000001e-06 2024-09-17T23:29:40.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464029+0000", 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2900000000000002e-06 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464031+0000", 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5629999999999999e-06 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23024 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464687+0000", 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0088481170000000008, 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014212558, 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23024 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464687+0000", 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464687+0000", 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464687+0000", 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464688+0000", 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.573e-06 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464692+0000", 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.929e-06 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464695+0000", 2024-09-17T23:29:40.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8320000000000002e-06 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23227 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466602+0000", 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.006932957, 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012352817, 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23227 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466602+0000", 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466602+0000", 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466602+0000", 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466605+0000", 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5780000000000001e-06 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466612+0000", 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5730000000000002e-06 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466629+0000", 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6612999999999998e-05 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23259 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472689+0000", 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00084614499999999997, 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063212839999999999, 2024-09-17T23:29:40.881 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23259 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472689+0000", 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472689+0000", 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472689+0000", 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472692+0000", 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.525e-06 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472696+0000", 2024-09-17T23:29:40.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1080000000000003e-06 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472699+0000", 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0630000000000002e-06 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23307 3.9 3.a1f7949 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.477836+0000", 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9956989, 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012307590000000001, 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23307 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477836+0000", 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477836+0000", 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477836+0000", 2024-09-17T23:29:40.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.27e-07 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477839+0000", 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8279999999999999e-06 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477843+0000", 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-17T23:29:40.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477846+0000", 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22961 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463383+0000", 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010152075999999999, 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015740190000000001, 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22961 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463383+0000", 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463383+0000", 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463383+0000", 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463384+0000", 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463387+0000", 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1020000000000001e-06 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463390+0000", 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0589999999999998e-06 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22993 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464034+0000", 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095004990000000008, 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015143733, 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22993 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464034+0000", 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464034+0000", 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464034+0000", 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1400000000000001e-07 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464036+0000", 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.114e-06 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464039+0000", 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6959999999999998e-06 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464042+0000", 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23025 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464699+0000", 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0088357039999999998, 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014533130999999999, 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23025 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464699+0000", 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464699+0000", 2024-09-17T23:29:40.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464699+0000", 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464700+0000", 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.141e-06 2024-09-17T23:29:40.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464704+0000", 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.101e-06 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464706+0000", 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23228 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466642+0000", 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0068929589999999997, 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012643639999999999, 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23228 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466642+0000", 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466642+0000", 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466642+0000", 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466645+0000", 2024-09-17T23:29:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6709999999999999e-06 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466652+0000", 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466661+0000", 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7270000000000002e-06 2024-09-17T23:29:40.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23260 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472703+0000", 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00083210599999999999, 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066373029999999998, 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23260 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472703+0000", 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472703+0000", 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472703+0000", 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2399999999999999e-07 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472704+0000", 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.395e-06 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472708+0000", 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3189999999999999e-06 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472710+0000", 2024-09-17T23:29:40.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6510000000000001e-06 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23308 3.9 3.a1f7949 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.477849+0000", 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9956851, 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015487680000000001, 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23308 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477849+0000", 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477849+0000", 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477850+0000", 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3799999999999999e-07 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477851+0000", 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.525e-06 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477855+0000", 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3000000000000002e-06 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477859+0000", 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1989999999999999e-06 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22962 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463393+0000", 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010141397, 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016062900000000001, 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22962 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463393+0000", 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463393+0000", 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463394+0000", 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463395+0000", 2024-09-17T23:29:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0550000000000001e-06 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463398+0000", 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1410000000000001e-06 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463401+0000", 2024-09-17T23:29:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3720000000000001e-06 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22994 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464045+0000", 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094900009999999996, 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015467046999999999, 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22994 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464045+0000", 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464045+0000", 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464045+0000", 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464046+0000", 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0580000000000001e-06 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464049+0000", 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0120000000000001e-06 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464053+0000", 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.726e-06 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23026 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464709+0000", 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0088253979999999999, 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01486265, 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23026 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464709+0000", 2024-09-17T23:29:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464709+0000", 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464710+0000", 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464711+0000", 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1209999999999999e-06 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464714+0000", 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9699999999999999e-06 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464716+0000", 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6240000000000002e-06 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23229 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466670+0000", 2024-09-17T23:29:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0068651069999999996, 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012956719, 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23229 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466670+0000", 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466670+0000", 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466670+0000", 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466672+0000", 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466680+0000", 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7389999999999999e-06 2024-09-17T23:29:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466686+0000", 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4219999999999999e-06 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23261 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472713+0000", 2024-09-17T23:29:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00082133399999999995, 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069691290000000001, 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23261 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472713+0000", 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472713+0000", 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472714+0000", 2024-09-17T23:29:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472719+0000", 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9899999999999997e-06 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472723+0000", 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1309999999999999e-06 2024-09-17T23:29:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472726+0000", 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23309 3.9 3.a1f7949 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.477877+0000", 2024-09-17T23:29:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9956574, 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001862601, 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23309 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477877+0000", 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477877+0000", 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477878+0000", 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7100000000000002e-07 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477881+0000", 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2380000000000002e-06 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477886+0000", 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9609999999999996e-06 2024-09-17T23:29:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477889+0000", 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.083e-06 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22963 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.463404+0000", 2024-09-17T23:29:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010130652, 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016391276, 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22963 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463404+0000", 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463404+0000", 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463404+0000", 2024-09-17T23:29:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-07 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463405+0000", 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.11e-06 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463409+0000", 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8229999999999997e-06 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.463412+0000", 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2380000000000002e-06 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22995 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464056+0000", 2024-09-17T23:29:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094790589999999997, 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015794268, 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22995 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464056+0000", 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464056+0000", 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464056+0000", 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-07 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464060+0000", 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5549999999999996e-06 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464064+0000", 2024-09-17T23:29:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3639999999999999e-06 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464067+0000", 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6349999999999999e-06 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23027 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464719+0000", 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.008815448, 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015184237999999999, 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23027 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464719+0000", 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464719+0000", 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464719+0000", 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464720+0000", 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.02e-06 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464723+0000", 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.914e-06 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464726+0000", 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5469999999999999e-06 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23230 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466715+0000", 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0068194199999999996, 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013243158999999999, 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23230 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466715+0000", 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466715+0000", 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466716+0000", 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466718+0000", 2024-09-17T23:29:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1919999999999999e-06 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466722+0000", 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5639999999999998e-06 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466726+0000", 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1879999999999998e-06 2024-09-17T23:29:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23262 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472729+0000", 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00080567200000000001, 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072872919999999999, 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23262 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472729+0000", 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472729+0000", 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472729+0000", 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4600000000000001e-07 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472731+0000", 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.232e-06 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472733+0000", 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-06 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472736+0000", 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23310 3.9 3.a1f7949 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.477894+0000", 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9956408, 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021559919999999998, 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23310 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477894+0000", 2024-09-17T23:29:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477894+0000", 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477894+0000", 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.27e-07 2024-09-17T23:29:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477896+0000", 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.516e-06 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477899+0000", 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.056e-06 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.477901+0000", 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.480043+0000", 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002141516 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.480050+0000", 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22996 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464071+0000", 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094639770000000002, 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016067504999999999, 2024-09-17T23:29:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22996 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464071+0000", 2024-09-17T23:29:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464071+0000", 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464071+0000", 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3900000000000001e-07 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464072+0000", 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1119999999999999e-06 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464076+0000", 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.155e-06 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464079+0000", 2024-09-17T23:29:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4660000000000002e-06 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23028 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464729+0000", 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0088057980000000001, 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015462632, 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23028 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464729+0000", 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464729+0000", 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464729+0000", 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464730+0000", 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0529999999999999e-06 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464734+0000", 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6040000000000001e-06 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464736+0000", 2024-09-17T23:29:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23231 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466730+0000", 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.006804381, 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013516333, 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23231 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466730+0000", 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466730+0000", 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466731+0000", 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4499999999999998e-07 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466732+0000", 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.282e-06 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466736+0000", 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3359999999999997e-06 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466739+0000", 2024-09-17T23:29:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23263 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472739+0000", 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000795694, 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075637009999999999, 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23263 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472739+0000", 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472739+0000", 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472739+0000", 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0100000000000001e-07 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472740+0000", 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.096e-06 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472743+0000", 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8839999999999998e-06 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472746+0000", 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6299999999999998e-06 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22997 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464082+0000", 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094531860000000006, 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016278010999999998, 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22997 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464082+0000", 2024-09-17T23:29:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464082+0000", 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464082+0000", 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:29:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464083+0000", 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.133e-06 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464086+0000", 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9969999999999999e-06 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464088+0000", 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4040000000000002e-06 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23029 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464740+0000", 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087945980000000007, 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015673011000000001, 2024-09-17T23:29:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23029 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464740+0000", 2024-09-17T23:29:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464740+0000", 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464740+0000", 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464741+0000", 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.083e-06 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464745+0000", 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.614e-06 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464748+0000", 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23232 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466742+0000", 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0067923710000000002, 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01372374, 2024-09-17T23:29:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23232 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466742+0000", 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466742+0000", 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466743+0000", 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-17T23:29:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466744+0000", 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1459999999999999e-06 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466747+0000", 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.055e-06 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466750+0000", 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2109999999999998e-06 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23264 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472749+0000", 2024-09-17T23:29:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00078601500000000004, 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007770239, 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23264 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472749+0000", 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472749+0000", 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472749+0000", 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472750+0000", 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1030000000000001e-06 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472753+0000", 2024-09-17T23:29:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.875e-06 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472757+0000", 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9389999999999999e-06 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22998 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464091+0000", 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094435790000000006, 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016489441000000001, 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22998 2024-09-17T23:29:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464091+0000", 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464091+0000", 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464091+0000", 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464092+0000", 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.06e-06 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464095+0000", 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.988e-06 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464098+0000", 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23030 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464751+0000", 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087840799999999997, 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015888238999999998, 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23030 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464751+0000", 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464751+0000", 2024-09-17T23:29:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464751+0000", 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0200000000000001e-07 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464755+0000", 2024-09-17T23:29:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6759999999999998e-06 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464759+0000", 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9400000000000004e-06 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464762+0000", 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6520000000000002e-06 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23233 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466753+0000", 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.006781614, 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013939579000000001, 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23233 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466753+0000", 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466753+0000", 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466844+0000", 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0538999999999995e-05 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466847+0000", 2024-09-17T23:29:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9569999999999998e-06 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466856+0000", 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8689999999999999e-06 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466863+0000", 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5819999999999998e-06 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23265 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472761+0000", 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00077411500000000002, 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079862849999999992, 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23265 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472761+0000", 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472761+0000", 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472761+0000", 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1899999999999999e-07 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472762+0000", 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1170000000000001e-06 2024-09-17T23:29:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472765+0000", 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472768+0000", 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7470000000000001e-06 2024-09-17T23:29:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22999 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464106+0000", 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094287520000000003, 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016696958000000001, 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22999 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464106+0000", 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464106+0000", 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464107+0000", 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9699999999999995e-07 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464109+0000", 2024-09-17T23:29:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0499999999999999e-06 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464119+0000", 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0723999999999999e-05 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464126+0000", 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.579e-06 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23031 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464765+0000", 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087693460000000008, 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016096605, 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23031 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464765+0000", 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464765+0000", 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464766+0000", 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1400000000000001e-07 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464767+0000", 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1000000000000001e-06 2024-09-17T23:29:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464774+0000", 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6769999999999995e-06 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464778+0000", 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9759999999999997e-06 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23234 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.466875+0000", 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0066597089999999998, 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014041991, 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23234 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466875+0000", 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.466875+0000", 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467074+0000", 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000198903 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467077+0000", 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.943e-06 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467083+0000", 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6270000000000002e-06 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467089+0000", 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6529999999999999e-06 2024-09-17T23:29:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23266 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472771+0000", 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00076343600000000004, 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081981970000000008, 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23266 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472771+0000", 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472771+0000", 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472772+0000", 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472773+0000", 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.189e-06 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472776+0000", 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0750000000000002e-06 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472778+0000", 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5349999999999999e-06 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23032 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464800+0000", 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087343709999999995, 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016224563000000001, 2024-09-17T23:29:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23032 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464800+0000", 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464800+0000", 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464801+0000", 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-07 2024-09-17T23:29:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464803+0000", 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.323e-06 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464811+0000", 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464816+0000", 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9690000000000002e-06 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23235 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.467097+0000", 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0064377920000000003, 2024-09-17T23:29:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013981278, 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23235 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467097+0000", 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467097+0000", 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467192+0000", 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5061000000000003e-05 2024-09-17T23:29:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467196+0000", 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8970000000000001e-06 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467204+0000", 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8620000000000003e-06 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467210+0000", 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5089999999999999e-06 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23267 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472782+0000", 2024-09-17T23:29:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00075317699999999995, 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083497680000000005, 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23267 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472782+0000", 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472782+0000", 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472782+0000", 2024-09-17T23:29:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472783+0000", 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1710000000000001e-06 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472786+0000", 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9280000000000002e-06 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472788+0000", 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22711 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479302+0000", 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9942322, 2024-09-17T23:29:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018878429999999999, 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22711 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479302+0000", 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479302+0000", 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479303+0000", 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6300000000000004e-07 2024-09-17T23:29:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479307+0000", 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479313+0000", 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3180000000000003e-06 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479317+0000", 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3050000000000003e-06 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23033 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.464826+0000", 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087090380000000005, 2024-09-17T23:29:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016420097000000002, 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23033 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464826+0000", 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464826+0000", 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464826+0000", 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464829+0000", 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.531e-06 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464840+0000", 2024-09-17T23:29:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.151e-05 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.464849+0000", 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.098e-06 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23236 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.467222+0000", 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0063125680000000002, 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014227394000000001, 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23236 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467222+0000", 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467222+0000", 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467350+0000", 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012798900000000001 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467353+0000", 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0189999999999998e-06 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467359+0000", 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2140000000000001e-06 2024-09-17T23:29:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467364+0000", 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.831e-06 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23268 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472792+0000", 2024-09-17T23:29:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00074221900000000002, 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087115319999999993, 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23268 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472792+0000", 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472792+0000", 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472793+0000", 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472795+0000", 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9800000000000001e-06 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472801+0000", 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:29:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472805+0000", 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22712 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479340+0000", 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9941945, 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002229122, 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22712 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479340+0000", 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479340+0000", 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479341+0000", 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479344+0000", 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7590000000000002e-06 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479351+0000", 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2350000000000002e-06 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479356+0000", 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0880000000000002e-06 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23237 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.467372+0000", 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0061631230000000004, 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014259104999999999, 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23237 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467372+0000", 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467372+0000", 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467468+0000", 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6272000000000003e-05 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467477+0000", 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1889999999999996e-06 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467486+0000", 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.721e-06 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467492+0000", 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4810000000000003e-06 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23269 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.472823+0000", 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000711486, 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088639679999999998, 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23269 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472823+0000", 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.472823+0000", 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474566+0000", 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017432470000000001 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474571+0000", 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5170000000000001e-06 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474580+0000", 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2429999999999994e-06 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474587+0000", 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6969999999999996e-06 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22713 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479364+0000", 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9941707, 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023772649999999999, 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22713 2024-09-17T23:29:41.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479364+0000", 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479364+0000", 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479364+0000", 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3700000000000001e-07 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479367+0000", 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479373+0000", 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0430000000000004e-06 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479377+0000", 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23238 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.467505+0000", 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0060301499999999997, 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014297363, 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23238 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467505+0000", 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467505+0000", 2024-09-17T23:29:41.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467634+0000", 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129942 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467638+0000", 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.461e-06 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467644+0000", 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1589999999999998e-06 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467648+0000", 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9609999999999999e-06 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23270 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474599+0000", 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9989352, 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072566310000000004, 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23270 2024-09-17T23:29:41.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474599+0000", 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474599+0000", 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474600+0000", 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474602+0000", 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4609999999999998e-06 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474609+0000", 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7900000000000002e-06 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474614+0000", 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22714 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479399+0000", 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9941359, 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025142699999999999, 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22714 2024-09-17T23:29:41.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479399+0000", 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479399+0000", 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479399+0000", 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479401+0000", 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.995e-06 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479406+0000", 2024-09-17T23:29:41.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9400000000000001e-06 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479409+0000", 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1870000000000001e-06 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23239 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.467655+0000", 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.005879291, 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014316363, 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23239 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467655+0000", 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467655+0000", 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467757+0000", 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010181199999999999 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467761+0000", 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.557e-06 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467768+0000", 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6399999999999997e-06 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.467776+0000", 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7300000000000005e-06 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23271 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474621+0000", 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9989133, 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074119809999999998, 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23271 2024-09-17T23:29:41.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474621+0000", 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474621+0000", 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474622+0000", 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474624+0000", 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.244e-06 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474631+0000", 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4520000000000002e-06 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474636+0000", 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22715 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479420+0000", 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9941149, 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026691079999999999, 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22715 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479420+0000", 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479420+0000", 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479420+0000", 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6800000000000002e-07 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479425+0000", 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7600000000000002e-06 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479429+0000", 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0369999999999996e-06 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479432+0000", 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7420000000000002e-06 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23272 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474642+0000", 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9988923, 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075033000000000001, 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23272 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474642+0000", 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474642+0000", 2024-09-17T23:29:41.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474643+0000", 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474645+0000", 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.137e-06 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474651+0000", 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9059999999999999e-06 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474656+0000", 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9609999999999996e-06 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22716 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479435+0000", 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9940991, 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027711089999999999, 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22716 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479435+0000", 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479435+0000", 2024-09-17T23:29:41.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479436+0000", 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479437+0000", 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.204e-06 2024-09-17T23:29:41.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479440+0000", 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479443+0000", 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6410000000000002e-06 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23273 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474663+0000", 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9988718, 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076017680000000001, 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23273 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474663+0000", 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474663+0000", 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474664+0000", 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474671+0000", 2024-09-17T23:29:41.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2239999999999998e-06 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474684+0000", 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2784e-05 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474687+0000", 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.771e-06 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22717 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479476+0000", 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9940591, 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028475990000000001, 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22717 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479476+0000", 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479476+0000", 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479476+0000", 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479477+0000", 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4529999999999999e-06 2024-09-17T23:29:41.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479481+0000", 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5489999999999998e-06 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479484+0000", 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23274 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474693+0000", 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9988418, 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076881240000000002, 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23274 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474693+0000", 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474693+0000", 2024-09-17T23:29:41.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474693+0000", 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1199999999999999e-07 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474695+0000", 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.761e-06 2024-09-17T23:29:41.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474699+0000", 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999998e-06 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474701+0000", 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6740000000000001e-06 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22718 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479503+0000", 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9940314, 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029361690000000002, 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22718 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479503+0000", 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479503+0000", 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479504+0000", 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479505+0000", 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.48e-06 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479509+0000", 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5030000000000002e-06 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479512+0000", 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3869999999999999e-06 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23275 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474705+0000", 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9988298, 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077929430000000001, 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23275 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474705+0000", 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474705+0000", 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474705+0000", 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474706+0000", 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2309999999999999e-06 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474709+0000", 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1599999999999998e-06 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474713+0000", 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1879999999999998e-06 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22719 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479515+0000", 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.994019, 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003043451, 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22719 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479515+0000", 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479515+0000", 2024-09-17T23:29:41.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479516+0000", 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.46e-07 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479517+0000", 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1790000000000001e-06 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479520+0000", 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2969999999999999e-06 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479524+0000", 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8449999999999996e-06 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:23276 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.474717+0000", 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9988179, 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079065880000000009, 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23276 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474717+0000", 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474717+0000", 2024-09-17T23:29:41.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474717+0000", 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.34e-07 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474718+0000", 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.198e-06 2024-09-17T23:29:41.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474721+0000", 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.101e-06 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.474724+0000", 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.576e-06 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22720 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479528+0000", 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9940066, 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031558189999999998, 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22720 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479528+0000", 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479528+0000", 2024-09-17T23:29:41.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479528+0000", 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479529+0000", 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1230000000000001e-06 2024-09-17T23:29:41.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479532+0000", 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9890000000000001e-06 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479535+0000", 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6220000000000001e-06 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22721 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479538+0000", 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9939966, 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032048419999999998, 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22721 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479538+0000", 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479538+0000", 2024-09-17T23:29:41.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479538+0000", 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479539+0000", 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1459999999999999e-06 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479542+0000", 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9550000000000001e-06 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479545+0000", 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5689999999999999e-06 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:22722 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.479575+0000", 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9939599, 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032296479999999999, 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22722 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479575+0000", 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479575+0000", 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479575+0000", 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479578+0000", 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479585+0000", 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4329999999999998e-06 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.479589+0000", 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3189999999999999e-06 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.145 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 160 2024-09-17T23:29:41.146 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:41.147 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:41.145+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T23:29:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:1122 read 0x37e9800 thru 0x37f8c45 (0xf446 bytes) 2024-09-17T23:29:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:1124 punch from 0xcb5b000 to 0xcb66200, (0xb200 bytes) 2024-09-17T23:29:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:1126 read 0xc4cae00 thru 0xc4cbe15 (0x1016 bytes) 2024-09-17T23:29:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:1128 write 0x6b81800 thru 0x6b90236 (0xea37 bytes) 2024-09-17T23:29:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:1129 punch from 0x19fc000 to 0x1a05c00, (0x9c00 bytes) 2024-09-17T23:29:41.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:41.248 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:41.245+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:41.256 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:41.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:41.348 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:41.346+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16300 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106014+0000", 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.306284929, 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043044675999999997, 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16300 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106014+0000", 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106014+0000", 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106015+0000", 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3000000000000002e-07 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106016+0000", 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.091e-06 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106019+0000", 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0460000000000001e-06 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106021+0000", 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.621e-06 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149051+0000", 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043029737999999998 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149059+0000", 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8499999999999994e-06 2024-09-17T23:29:41.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16301 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106025+0000", 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.30627468, 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043049500999999997, 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16301 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106025+0000", 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106025+0000", 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106025+0000", 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-07 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106026+0000", 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9099999999999991e-07 2024-09-17T23:29:41.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106029+0000", 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.937e-06 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106031+0000", 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149069+0000", 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043038248000000001 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149074+0000", 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5449999999999997e-06 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16302 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106034+0000", 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3062650220000001, 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043053632000000001, 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16302 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106034+0000", 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106034+0000", 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106034+0000", 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106043+0000", 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5550000000000003e-06 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106052+0000", 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8130000000000002e-06 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106055+0000", 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0120000000000001e-06 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149083+0000", 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043027867999999997 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149088+0000", 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16303 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106059+0000", 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3062401399999999, 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043048824999999999, 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16303 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106059+0000", 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106059+0000", 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106059+0000", 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4999999999999999e-07 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106061+0000", 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106067+0000", 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7270000000000001e-06 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106070+0000", 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7640000000000001e-06 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149104+0000", 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043033559999999998 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149108+0000", 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3329999999999998e-06 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16304 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106073+0000", 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.30622587, 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043055359000000001, 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16304 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106073+0000", 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106073+0000", 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106074+0000", 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4999999999999999e-07 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106075+0000", 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.15e-06 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106078+0000", 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.066e-06 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106080+0000", 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149124+0000", 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043043792999999997 2024-09-17T23:29:41.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149129+0000", 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5170000000000001e-06 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16305 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106083+0000", 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.306215884, 2024-09-17T23:29:41.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043065610999999997, 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16305 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106083+0000", 2024-09-17T23:29:41.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106083+0000", 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106084+0000", 2024-09-17T23:29:41.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106085+0000", 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3319999999999999e-06 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106089+0000", 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-06 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106092+0000", 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000001e-06 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149144+0000", 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043052199999999999 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149149+0000", 2024-09-17T23:29:41.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4870000000000004e-06 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16306 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106099+0000", 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3062000679999999, 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043070275999999998, 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16306 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106099+0000", 2024-09-17T23:29:41.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106099+0000", 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106100+0000", 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.89e-07 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106102+0000", 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8989999999999999e-06 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106109+0000", 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106114+0000", 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0499999999999999e-06 2024-09-17T23:29:41.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149165+0000", 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043051007000000002 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149169+0000", 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4039999999999997e-06 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16307 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106123+0000", 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3061764490000001, 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043066276000000001, 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16307 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106123+0000", 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106123+0000", 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106123+0000", 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:29:41.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106125+0000", 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106137+0000", 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1945e-05 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106144+0000", 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4590000000000001e-06 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149185+0000", 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043040228999999999 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149189+0000", 2024-09-17T23:29:41.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3170000000000003e-06 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16308 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106154+0000", 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.306145565, 2024-09-17T23:29:41.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043055641999999998, 2024-09-17T23:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16308 2024-09-17T23:29:41.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106154+0000", 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106154+0000", 2024-09-17T23:29:41.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106154+0000", 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106156+0000", 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3089999999999998e-06 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106165+0000", 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2670000000000006e-06 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106171+0000", 2024-09-17T23:29:41.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3049999999999998e-06 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149205+0000", 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043034138 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149209+0000", 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16309 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106178+0000", 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3061211639999999, 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043051566999999999, 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16309 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106178+0000", 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106178+0000", 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106179+0000", 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.63e-07 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106181+0000", 2024-09-17T23:29:41.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106189+0000", 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2500000000000006e-06 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106194+0000", 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149225+0000", 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043030959000000001 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149230+0000", 2024-09-17T23:29:41.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16310 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106201+0000", 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.306098191, 2024-09-17T23:29:41.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043063279000000003, 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16310 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106201+0000", 2024-09-17T23:29:41.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106201+0000", 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106201+0000", 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2800000000000002e-07 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106204+0000", 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1040000000000001e-06 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106211+0000", 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8269999999999998e-06 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106217+0000", 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5380000000000002e-06 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149253+0000", 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043036199999999997 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149264+0000", 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1182000000000001e-05 2024-09-17T23:29:41.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16311 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106227+0000", 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.306071939, 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043054773999999997, 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16311 2024-09-17T23:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106227+0000", 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106227+0000", 2024-09-17T23:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106228+0000", 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2799999999999996e-07 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106230+0000", 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3970000000000001e-06 2024-09-17T23:29:41.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106237+0000", 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1080000000000004e-06 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106242+0000", 2024-09-17T23:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9049999999999996e-06 2024-09-17T23:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149277+0000", 2024-09-17T23:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043034848000000001 2024-09-17T23:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149282+0000", 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8879999999999996e-06 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.479 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16312 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106249+0000", 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.306050224, 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043044647999999998, 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16312 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106249+0000", 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106249+0000", 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106249+0000", 2024-09-17T23:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.34e-07 2024-09-17T23:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106251+0000", 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106258+0000", 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3470000000000004e-06 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106263+0000", 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0860000000000001e-06 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149290+0000", 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043026920000000003 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149294+0000", 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8210000000000003e-06 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16313 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106283+0000", 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3060162929999999, 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043036820000000003, 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16313 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106283+0000", 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106283+0000", 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106283+0000", 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106286+0000", 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4530000000000001e-06 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106292+0000", 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5579999999999997e-06 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106298+0000", 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8690000000000001e-06 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149310+0000", 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043011981999999997 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149320+0000", 2024-09-17T23:29:41.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4700000000000008e-06 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16314 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106319+0000", 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3059803030000001, 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043033367000000003, 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16314 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106319+0000", 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106319+0000", 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106319+0000", 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-07 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106323+0000", 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0479999999999999e-06 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106333+0000", 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0761000000000001e-05 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106340+0000", 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4999999999999996e-06 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149347+0000", 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043006996999999998 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149352+0000", 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5090000000000001e-06 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16315 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106352+0000", 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3059468940000001, 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043032481999999997, 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16315 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106352+0000", 2024-09-17T23:29:41.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106352+0000", 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106353+0000", 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106354+0000", 2024-09-17T23:29:41.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.84e-06 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106359+0000", 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7450000000000003e-06 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106362+0000", 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0019999999999998e-06 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149376+0000", 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043013939000000001 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149385+0000", 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6139999999999992e-06 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16316 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106366+0000", 2024-09-17T23:29:41.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3059331009999999, 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043055094000000002, 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16316 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106366+0000", 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106366+0000", 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106366+0000", 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1399999999999998e-07 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106368+0000", 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.142e-06 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106374+0000", 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9789999999999998e-06 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106376+0000", 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149414+0000", 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043037432 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149421+0000", 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5089999999999996e-06 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16317 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106380+0000", 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3059190680000001, 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043075582000000001, 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16317 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106380+0000", 2024-09-17T23:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106380+0000", 2024-09-17T23:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106380+0000", 2024-09-17T23:29:41.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.35e-07 2024-09-17T23:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106382+0000", 2024-09-17T23:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3319999999999999e-06 2024-09-17T23:29:41.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106386+0000", 2024-09-17T23:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8670000000000004e-06 2024-09-17T23:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106388+0000", 2024-09-17T23:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6579999999999998e-06 2024-09-17T23:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149446+0000", 2024-09-17T23:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043057441000000002 2024-09-17T23:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149456+0000", 2024-09-17T23:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0049e-05 2024-09-17T23:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16318 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106391+0000", 2024-09-17T23:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.305907742, 2024-09-17T23:29:41.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043079124000000003, 2024-09-17T23:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16318 2024-09-17T23:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106391+0000", 2024-09-17T23:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106391+0000", 2024-09-17T23:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106392+0000", 2024-09-17T23:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106393+0000", 2024-09-17T23:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1039999999999999e-06 2024-09-17T23:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106396+0000", 2024-09-17T23:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9620000000000002e-06 2024-09-17T23:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106398+0000", 2024-09-17T23:29:41.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.649e-06 2024-09-17T23:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149466+0000", 2024-09-17T23:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043067673000000001 2024-09-17T23:29:41.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149471+0000", 2024-09-17T23:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.532e-06 2024-09-17T23:29:41.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:16319 3.15 3.a68d86b5 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.106401+0000", 2024-09-17T23:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.305897877, 2024-09-17T23:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043093587000000003, 2024-09-17T23:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 16319 2024-09-17T23:29:41.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106401+0000", 2024-09-17T23:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106401+0000", 2024-09-17T23:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106546+0000", 2024-09-17T23:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014462200000000001 2024-09-17T23:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106549+0000", 2024-09-17T23:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4580000000000002e-06 2024-09-17T23:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106558+0000", 2024-09-17T23:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6619999999999994e-06 2024-09-17T23:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.106572+0000", 2024-09-17T23:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3875e-05 2024-09-17T23:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149485+0000", 2024-09-17T23:29:41.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04291329 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.149495+0000", 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6800000000000005e-06 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.531 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:41.532 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:41.530+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:41.533 INFO:teuthology.orchestra.run.smithi139.stdout:1130 write 0xebe4000 thru 0xebe4f0d (0xf0e bytes) 2024-09-17T23:29:41.533 INFO:teuthology.orchestra.run.smithi139.stdout:1132 write 0xb7e6400 thru 0xb7e934b (0x2f4c bytes) 2024-09-17T23:29:41.533 INFO:teuthology.orchestra.run.smithi139.stdout:1133 trunc from 0xed20400 to 0x7db5a00 2024-09-17T23:29:41.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:41.633 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:41.631+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:41.646 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:41.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:41.734 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:41.732+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:41.765 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:41.766 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:41.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:41.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:41.862 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:41.859+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.422560828999998, 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:41.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:41.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:41.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1061 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 657408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4494936599999999, 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037158380999999997, 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1061 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:41.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444509+0000", 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7129999999999999e-06 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444512+0000", 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2059999999999999e-06 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444515+0000", 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481580+0000", 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037065396 2024-09-17T23:29:41.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481625+0000", 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5049000000000002e-05 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481665+0000", 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9949e-05 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1062 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 666624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4494825640000002, 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037282721999999997, 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1062 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0100000000000001e-07 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444519+0000", 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0219999999999999e-06 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444522+0000", 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1360000000000001e-06 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444526+0000", 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6950000000000002e-06 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481709+0000", 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037183131000000001 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481761+0000", 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1768000000000003e-05 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481801+0000", 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9768999999999998e-05 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1063 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 675840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.449471494, 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037384175999999998, 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1063 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444784+0000", 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025496399999999999 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444788+0000", 2024-09-17T23:29:41.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0439999999999997e-06 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444794+0000", 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5019999999999998e-06 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444798+0000", 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8959999999999996e-06 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481829+0000", 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037030346999999998 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481875+0000", 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6490999999999999e-05 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481913+0000", 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7932000000000002e-05 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.919 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1064 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 685056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4491897759999999, 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037215831999999997, 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1064 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7700000000000001e-07 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444821+0000", 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.046e-05 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444826+0000", 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7380000000000002e-06 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444829+0000", 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2169999999999999e-06 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481946+0000", 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037116984999999998 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481989+0000", 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589999999999997e-05 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482026+0000", 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7564999999999997e-05 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1065 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 694272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4491502570000003, 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037301835999999998, 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1065 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444851+0000", 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444852+0000", 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8819999999999999e-06 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444856+0000", 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0849999999999999e-06 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444859+0000", 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9919999999999999e-06 2024-09-17T23:29:41.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482057+0000", 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037197539000000002 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482107+0000", 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9530000000000002e-05 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482152+0000", 2024-09-17T23:29:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5419e-05 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1066 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 703488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4491362099999998, 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037415125, 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1066 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:41.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445344+0000", 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047944399999999999 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445346+0000", 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445359+0000", 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2573e-05 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445363+0000", 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482181+0000", 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036818371000000003 2024-09-17T23:29:41.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482225+0000", 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3591000000000003e-05 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482279+0000", 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4447000000000002e-05 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1069 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 731136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4485890289999999, 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037253597999999999, 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1069 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445412+0000", 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.72e-07 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445413+0000", 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2780000000000001e-06 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445416+0000", 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3529999999999999e-06 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445425+0000", 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0119999999999994e-06 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482567+0000", 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037141277 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482618+0000", 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1445e-05 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482665+0000", 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6961e-05 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1070 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 740352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4485579370000004, 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037344291000000002, 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1070 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445443+0000", 2024-09-17T23:29:41.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445446+0000", 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2480000000000001e-06 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445452+0000", 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5659999999999997e-06 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445455+0000", 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482693+0000", 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037238009000000002 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482745+0000", 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2382000000000001e-05 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482787+0000", 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1337000000000001e-05 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1071 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4485402460000003, 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037450310000000001, 2024-09-17T23:29:41.943 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1071 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445528+0000", 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7896000000000002e-05 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445530+0000", 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2630000000000002e-06 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445536+0000", 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9660000000000001e-06 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445541+0000", 2024-09-17T23:29:41.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482816+0000", 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037275413 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482864+0000", 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7750999999999997e-05 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482910+0000", 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6249999999999999e-05 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1072 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 758784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4484442360000003, 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037482326000000003, 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1072 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445635+0000", 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9115000000000004e-05 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445638+0000", 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.779e-06 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445643+0000", 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4410000000000001e-06 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445647+0000", 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.794e-06 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482937+0000", 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037289309 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482988+0000", 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1374e-05 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483038+0000", 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0513999999999998e-05 2024-09-17T23:29:41.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1073 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4483452689999998, 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037507367, 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1073 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:41.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445741+0000", 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5687999999999997e-05 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445743+0000", 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6199999999999999e-06 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445749+0000", 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4319999999999998e-06 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445752+0000", 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.568e-06 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483069+0000", 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037316301000000003 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483114+0000", 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999000000000001e-05 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483162+0000", 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8758999999999999e-05 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1074 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4482418030000002, 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037534672999999998, 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1074 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445855+0000", 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6780999999999994e-05 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445858+0000", 2024-09-17T23:29:41.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6599999999999999e-06 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445864+0000", 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445869+0000", 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7400000000000004e-06 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483192+0000", 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037322704999999998 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483244+0000", 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2744999999999997e-05 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483293+0000", 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8792000000000003e-05 2024-09-17T23:29:41.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1075 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4481247259999996, 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037543656000000002, 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:41.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446843+0000", 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096762300000000003 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446852+0000", 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9439999999999994e-06 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446861+0000", 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.541e-06 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446871+0000", 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0498e-05 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483324+0000", 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036452248999999999 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483375+0000", 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1780000000000002e-05 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483419+0000", 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020999999999997e-05 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1083 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 860160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4468569950000001, 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037242256000000001, 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1083 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:41.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447144+0000", 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.16e-07 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447151+0000", 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9450000000000004e-06 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447154+0000", 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8639999999999998e-06 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447159+0000", 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484301+0000", 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037141835999999998 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484347+0000", 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5998000000000002e-05 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484386+0000", 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9060000000000002e-05 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1084 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 869376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4468362710000005, 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037333979000000003, 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1084 2024-09-17T23:29:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.67e-07 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447166+0000", 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2500000000000001e-06 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447169+0000", 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3189999999999999e-06 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447172+0000", 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484414+0000", 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037242311 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484458+0000", 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4515000000000003e-05 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484498+0000", 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9703000000000003e-05 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1085 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4468151240000005, 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037436900000000002, 2024-09-17T23:29:41.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1085 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447300+0000", 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011436 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447304+0000", 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2780000000000004e-06 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447312+0000", 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3440000000000001e-06 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447329+0000", 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6307000000000002e-05 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484526+0000", 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037196976999999999 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484581+0000", 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5019999999999998e-05 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484622+0000", 2024-09-17T23:29:41.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1613999999999998e-05 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1086 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 887808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4466534939999995, 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03739141, 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1086 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:41.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447532+0000", 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018467099999999999 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447536+0000", 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0899999999999998e-06 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447544+0000", 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2859999999999999e-06 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447565+0000", 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0706000000000001e-05 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484652+0000", 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037086898 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484696+0000", 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4159000000000002e-05 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484738+0000", 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-05 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1087 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 897024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4464224010000004, 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037273000000000001, 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1087 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447711+0000", 2024-09-17T23:29:41.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013340800000000001 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447715+0000", 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9929999999999997e-06 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447724+0000", 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6009999999999995e-06 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447730+0000", 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.852e-06 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484768+0000", 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037037887999999998 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484811+0000", 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3736999999999998e-05 2024-09-17T23:29:41.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484851+0000", 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9521000000000003e-05 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1088 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 906240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4462503370000004, 2024-09-17T23:29:41.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037212453999999999, 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1088 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447893+0000", 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000142993 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447901+0000", 2024-09-17T23:29:41.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8930000000000005e-06 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447907+0000", 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3300000000000004e-06 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447911+0000", 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1910000000000002e-06 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484880+0000", 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036968207000000003 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484923+0000", 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3197e-05 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484962+0000", 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9643e-05 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:41.996 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:41.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:29:41.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:41.998 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:41.999 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:41.999 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:41.994+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:42.012 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:42.012 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:42.038 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:42.067 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:42.067 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:42.094 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:42.094 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:42.098 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:42.096+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:42.148 INFO:teuthology.orchestra.run.smithi139.stdout:1134 read 0x2ec1c00 thru 0x2ec971a (0x7b1b bytes) 2024-09-17T23:29:42.163 INFO:teuthology.orchestra.run.smithi139.stdout:1137 write 0xba2a200 thru 0xba31a25 (0x7826 bytes) 2024-09-17T23:29:42.166 INFO:teuthology.orchestra.run.smithi139.stdout:1138 clone 11 order 21 su 262144 sc 14 2024-09-17T23:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.176 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:42.199 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:42.197+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:42.292 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.293 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.300 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:42.298+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:42.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:42.401 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:42.399+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.408 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.417 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:42.501 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:42.499+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:42.527 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.527 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:42.602 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:42.599+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:42.684 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.684 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:42.736 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:42.734+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:42.799 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.799 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.804 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:42.801+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:42.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:42.905 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:42.903+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:42.955 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:42.955 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:42.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:42.968 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:29:43.005 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:43.003+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:43.072 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone10 from 0xba31c00 (overlap 0x7db5a00) to 0x0 2024-09-17T23:29:43.106 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:43.104+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:43.118 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.118 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:43.207 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:43.205+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:43.277 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.278 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:43.308 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:43.306+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:43.399 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.400 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:43.409 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:43.408+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:43.450 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T23:29:43.510 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:43.508+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:43.544 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.544 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:43.641 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:43.639+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:43.712 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:43.710+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:43.725 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.725 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:43.813 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:43.811+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:43.852 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.853 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:43.867 INFO:teuthology.orchestra.run.smithi139.stdout:1141 read 0x10f6600 thru 0x1103487 (0xce88 bytes) 2024-09-17T23:29:43.880 INFO:teuthology.orchestra.run.smithi139.stdout:1143 write 0xc05c600 thru 0xc06263b (0x603c bytes) 2024-09-17T23:29:43.889 INFO:teuthology.orchestra.run.smithi139.stdout:1144 read 0x51f5a00 thru 0x52041a6 (0xe7a7 bytes) 2024-09-17T23:29:43.890 INFO:teuthology.orchestra.run.smithi139.stdout:1146 read 0x356e600 thru 0x3575aea (0x74eb bytes) 2024-09-17T23:29:43.891 INFO:teuthology.orchestra.run.smithi139.stdout:1148 punch from 0x40f0a00 to 0x40fd800, (0xce00 bytes) 2024-09-17T23:29:43.892 INFO:teuthology.orchestra.run.smithi139.stdout:1150 punch from 0xb3cd600 to 0xb3d4400, (0x6e00 bytes) 2024-09-17T23:29:43.892 INFO:teuthology.orchestra.run.smithi139.stdout:1152 write 0x6f86000 thru 0x6f94a5e (0xea5f bytes) 2024-09-17T23:29:43.892 INFO:teuthology.orchestra.run.smithi139.stdout:1153 write 0x9a40800 thru 0x9a463f2 (0x5bf3 bytes) 2024-09-17T23:29:43.894 INFO:teuthology.orchestra.run.smithi139.stdout:1154 read 0xa506000 thru 0xa510010 (0xa011 bytes) 2024-09-17T23:29:43.895 INFO:teuthology.orchestra.run.smithi139.stdout:1156 write 0x9a2f400 thru 0x9a3ea83 (0xf684 bytes) 2024-09-17T23:29:43.895 INFO:teuthology.orchestra.run.smithi139.stdout:1157 punch from 0x4260400 to 0x4268400, (0x8000 bytes) 2024-09-17T23:29:43.896 INFO:teuthology.orchestra.run.smithi139.stdout:1158 read 0x7ef5c00 thru 0x7f00398 (0xa799 bytes) 2024-09-17T23:29:43.901 INFO:teuthology.orchestra.run.smithi139.stdout:1160 trunc from 0xc062800 to 0xe98de00 2024-09-17T23:29:43.914 INFO:teuthology.orchestra.run.smithi139.stdout:1161 read 0xd8c6000 thru 0xd8d1e34 (0xbe35 bytes) 2024-09-17T23:29:43.914 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:43.912+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:43.915 INFO:teuthology.orchestra.run.smithi139.stdout:1162 write 0x630e400 thru 0x6317616 (0x9217 bytes) 2024-09-17T23:29:43.915 INFO:teuthology.orchestra.run.smithi139.stdout:1165 write 0x23e0000 thru 0x23ef88a (0xf88b bytes) 2024-09-17T23:29:43.915 INFO:teuthology.orchestra.run.smithi139.stdout:1166 read 0x951e000 thru 0x952b691 (0xd692 bytes) 2024-09-17T23:29:43.916 INFO:teuthology.orchestra.run.smithi139.stdout:1168 write 0xc800200 thru 0xc80d1e2 (0xcfe3 bytes) 2024-09-17T23:29:43.916 INFO:teuthology.orchestra.run.smithi139.stdout:1169 read 0x2c73600 thru 0x2c792cd (0x5cce bytes) 2024-09-17T23:29:43.916 INFO:teuthology.orchestra.run.smithi139.stdout:1171 read 0x242de00 thru 0x243a2fc (0xc4fd bytes) 2024-09-17T23:29:43.917 INFO:teuthology.orchestra.run.smithi139.stdout:1175 read 0x218f400 thru 0x2199c7f (0xa880 bytes) 2024-09-17T23:29:43.920 INFO:teuthology.orchestra.run.smithi139.stdout:1176 trunc from 0xe98de00 to 0xc97b000 2024-09-17T23:29:43.951 INFO:teuthology.orchestra.run.smithi139.stdout:1177 trunc from 0xc97b000 to 0x50f6800 2024-09-17T23:29:43.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:43.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:43.971 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T23:29:43.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:29:43.985 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:43.985 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:43.992 INFO:teuthology.orchestra.run.smithi139.stdout:1178 write 0x89ca000 thru 0x89d2f39 (0x8f3a bytes) 2024-09-17T23:29:43.995 INFO:teuthology.orchestra.run.smithi139.stdout:1179 trunc from 0x89d3000 to 0x6381000 2024-09-17T23:29:43.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:44.019 INFO:teuthology.orchestra.run.smithi139.stdout:1181 read 0x2505c00 thru 0x2508a4e (0x2e4f bytes) 2024-09-17T23:29:44.019 INFO:teuthology.orchestra.run.smithi139.stdout:1186 write 0x4d4400 thru 0x4d9c30 (0x5831 bytes) 2024-09-17T23:29:44.020 INFO:teuthology.orchestra.run.smithi139.stdout:1188 read 0x3245200 thru 0x3249b26 (0x4927 bytes) 2024-09-17T23:29:44.020 INFO:teuthology.orchestra.run.smithi139.stdout:1189 write 0x5f15c00 thru 0x5f16525 (0x926 bytes) 2024-09-17T23:29:44.038 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:44.035+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:44.044 INFO:teuthology.orchestra.run.smithi139.stdout:1190 write 0xdbdaa00 thru 0xdbde2b3 (0x38b4 bytes) 2024-09-17T23:29:44.045 INFO:teuthology.orchestra.run.smithi139.stdout:1195 read 0x5e44800 thru 0x5e488c1 (0x40c2 bytes) 2024-09-17T23:29:44.045 INFO:teuthology.orchestra.run.smithi139.stdout:1200 write 0x4145800 thru 0x4151e98 (0xc699 bytes) 2024-09-17T23:29:44.045 INFO:teuthology.orchestra.run.smithi139.stdout:1202 read 0x70fc00 thru 0x711527 (0x1928 bytes) 2024-09-17T23:29:44.046 INFO:teuthology.orchestra.run.smithi139.stdout:1203 read 0x3b72c00 thru 0x3b8285a (0xfc5b bytes) 2024-09-17T23:29:44.049 INFO:teuthology.orchestra.run.smithi139.stdout:1204 read 0x839dc00 thru 0x83a35a1 (0x59a2 bytes) 2024-09-17T23:29:44.059 INFO:teuthology.orchestra.run.smithi139.stdout:1205 trunc from 0xdbde400 to 0x56aca00 2024-09-17T23:29:44.083 INFO:teuthology.orchestra.run.smithi139.stdout:1207 read 0x4d1200 thru 0x4ded70 (0xdb71 bytes) 2024-09-17T23:29:44.084 INFO:teuthology.orchestra.run.smithi139.stdout:1210 read 0x50acc00 thru 0x50b0263 (0x3664 bytes) 2024-09-17T23:29:44.085 INFO:teuthology.orchestra.run.smithi139.stdout:1212 trunc from 0x56aca00 to 0x5d88c00 2024-09-17T23:29:44.100 INFO:teuthology.orchestra.run.smithi139.stdout:1213 read 0x293fe00 thru 0x2943a7b (0x3c7c bytes) 2024-09-17T23:29:44.101 INFO:teuthology.orchestra.run.smithi139.stdout:1214 punch from 0xfea00 to 0x101e00, (0x3400 bytes) 2024-09-17T23:29:44.102 INFO:teuthology.orchestra.run.smithi139.stdout:1215 trunc from 0x5d88c00 to 0x5663600 2024-09-17T23:29:44.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:44.116 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:44.114+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:44.130 INFO:teuthology.orchestra.run.smithi139.stdout:1217 trunc from 0x5663600 to 0xbb9f000 2024-09-17T23:29:44.139 INFO:teuthology.orchestra.run.smithi139.stdout:1219 write 0xa46b200 thru 0xa478264 (0xd065 bytes) 2024-09-17T23:29:44.144 INFO:teuthology.orchestra.run.smithi139.stdout:1220 read 0x4caa000 thru 0x4cac22c (0x222d bytes) 2024-09-17T23:29:44.145 INFO:teuthology.orchestra.run.smithi139.stdout:1221 punch from 0x8e52200 to 0x8e56400, (0x4200 bytes) 2024-09-17T23:29:44.145 INFO:teuthology.orchestra.run.smithi139.stdout:1222 punch from 0xb4b3c00 to 0xb4bbe00, (0x8200 bytes) 2024-09-17T23:29:44.160 INFO:teuthology.orchestra.run.smithi139.stdout:1223 write 0xd097000 thru 0xd0a33e3 (0xc3e4 bytes) 2024-09-17T23:29:44.160 INFO:teuthology.orchestra.run.smithi139.stdout:1224 write 0xa71c400 thru 0xa7241b3 (0x7db4 bytes) 2024-09-17T23:29:44.161 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:44.161 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:44.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:44.173 INFO:teuthology.orchestra.run.smithi139.stdout:1225 write 0xd816800 thru 0xd825137 (0xe938 bytes) 2024-09-17T23:29:44.174 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-17T23:29:44.187 INFO:teuthology.orchestra.run.smithi139.stdout:1228 trunc from 0xd825200 to 0x1c34000 2024-09-17T23:29:44.216 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:44.214+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:44.229 INFO:teuthology.orchestra.run.smithi139.stdout:1230 write 0x1c8f400 thru 0x1c92c89 (0x388a bytes) 2024-09-17T23:29:44.250 INFO:teuthology.orchestra.run.smithi139.stdout:1231 write 0x96f3000 thru 0x96ff490 (0xc491 bytes) 2024-09-17T23:29:44.252 INFO:teuthology.orchestra.run.smithi139.stdout:1234 clone 12 order 23 su 32768 sc 12 2024-09-17T23:29:44.252 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:44.252 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:44.289 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:44.289 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:44.293 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:44.293 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:44.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:44.310 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:44.311 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:44.327 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:44.327 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:44.328 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:44.325+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:44.332 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:44.332 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 42, 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:29:42.963335+0000", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:28:57.456516+0000", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:29:26.866978+0000", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:29:44.333 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:29:44.334 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:29:44.335 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:29:44.336 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:29:44.337 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:29:44.338 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:29:44.339 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:29:44.340 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:29:44.341 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:29:44.342 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:29:44.343 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:29:44.344 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:29:44.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:29:44.346 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5, 2024-09-17T23:29:44.347 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:29:44.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:29:44.349 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "42", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:29:44.350 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 13, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 42, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:29:44.351 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:29:44.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:29:44.353 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.8799999952316284, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.8799999952316284, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:29:44.354 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:44.357 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.358 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:29:44.360 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 36, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:44.364 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:29:44.365 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:44.366 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:29:44.367 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:29:44.371 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 36, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:29:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:29:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:29:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:29:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:29:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:29:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:29:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1870185623", 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1870185623", 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1870185623", 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1870185623", 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:29:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:29:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:29:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:29:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.397 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:44.398 INFO:tasks.thrashosds.thrasher:j is {'epoch': 42, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:29:42.963335+0000', 'last_up_change': '2024-09-17T23:28:57.456516+0000', 'last_in_change': '2024-09-17T23:29:26.866978+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 13, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1870185623}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1870185623}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1870185623}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1870185623}]}, 'public_addr': '172.21.15.37:6809/1870185623', 'cluster_addr': '172.21.15.37:6811/1870185623', 'heartbeat_back_addr': '172.21.15.37:6815/1870185623', 'heartbeat_front_addr': '172.21.15.37:6813/1870185623', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:29:44.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:29:44.399 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:44.400 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:44.419 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:44.417+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:44.420 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:44.421 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:44.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:44.520 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:44.518+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:44.580 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:44.580 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:44.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.106215081000002, 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:44.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:44.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:44.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:44.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1061 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 657408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1331479120000001, 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037158380999999997, 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1061 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:44.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444509+0000", 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7129999999999999e-06 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444512+0000", 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2059999999999999e-06 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444515+0000", 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481580+0000", 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037065396 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481625+0000", 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5049000000000002e-05 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481665+0000", 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9949e-05 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1062 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 666624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1331368160000004, 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037282721999999997, 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1062 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0100000000000001e-07 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444519+0000", 2024-09-17T23:29:44.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0219999999999999e-06 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444522+0000", 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1360000000000001e-06 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444526+0000", 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6950000000000002e-06 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481709+0000", 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037183131000000001 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481761+0000", 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1768000000000003e-05 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481801+0000", 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9768999999999998e-05 2024-09-17T23:29:44.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1063 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 675840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1331257460000002, 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037384175999999998, 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1063 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444784+0000", 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025496399999999999 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444788+0000", 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0439999999999997e-06 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444794+0000", 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5019999999999998e-06 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444798+0000", 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8959999999999996e-06 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481829+0000", 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037030346999999998 2024-09-17T23:29:44.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481875+0000", 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6490999999999999e-05 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481913+0000", 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7932000000000002e-05 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1064 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 685056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1328440280000001, 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037215831999999997, 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1064 2024-09-17T23:29:44.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7700000000000001e-07 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444821+0000", 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.046e-05 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444826+0000", 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7380000000000002e-06 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444829+0000", 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2169999999999999e-06 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481946+0000", 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037116984999999998 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481989+0000", 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589999999999997e-05 2024-09-17T23:29:44.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482026+0000", 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7564999999999997e-05 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1065 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 694272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1328045089999996, 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037301835999999998, 2024-09-17T23:29:44.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1065 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444851+0000", 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444852+0000", 2024-09-17T23:29:44.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8819999999999999e-06 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444856+0000", 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0849999999999999e-06 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444859+0000", 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9919999999999999e-06 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482057+0000", 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037197539000000002 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482107+0000", 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9530000000000002e-05 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482152+0000", 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5419e-05 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1066 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 703488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.132790462, 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037415125, 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1066 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445344+0000", 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047944399999999999 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445346+0000", 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445359+0000", 2024-09-17T23:29:44.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2573e-05 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445363+0000", 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482181+0000", 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036818371000000003 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482225+0000", 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3591000000000003e-05 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482279+0000", 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4447000000000002e-05 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1069 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 731136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.132243281, 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037253597999999999, 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1069 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445412+0000", 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.72e-07 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445413+0000", 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2780000000000001e-06 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445416+0000", 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3529999999999999e-06 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445425+0000", 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0119999999999994e-06 2024-09-17T23:29:44.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482567+0000", 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037141277 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482618+0000", 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1445e-05 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482665+0000", 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6961e-05 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1070 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 740352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1322121889999996, 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037344291000000002, 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1070 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445443+0000", 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445446+0000", 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2480000000000001e-06 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445452+0000", 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5659999999999997e-06 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445455+0000", 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482693+0000", 2024-09-17T23:29:44.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037238009000000002 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482745+0000", 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2382000000000001e-05 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482787+0000", 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1337000000000001e-05 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1071 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1321944979999996, 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037450310000000001, 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1071 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445528+0000", 2024-09-17T23:29:44.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7896000000000002e-05 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445530+0000", 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2630000000000002e-06 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445536+0000", 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9660000000000001e-06 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445541+0000", 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482816+0000", 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037275413 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482864+0000", 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7750999999999997e-05 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482910+0000", 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6249999999999999e-05 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1072 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 758784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1320984879999996, 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037482326000000003, 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1072 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445635+0000", 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9115000000000004e-05 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445638+0000", 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.779e-06 2024-09-17T23:29:44.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445643+0000", 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4410000000000001e-06 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445647+0000", 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.794e-06 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482937+0000", 2024-09-17T23:29:44.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037289309 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482988+0000", 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1374e-05 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483038+0000", 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0513999999999998e-05 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1073 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.131999521, 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037507367, 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1073 2024-09-17T23:29:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445741+0000", 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5687999999999997e-05 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445743+0000", 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6199999999999999e-06 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445749+0000", 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4319999999999998e-06 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445752+0000", 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.568e-06 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483069+0000", 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037316301000000003 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483114+0000", 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999000000000001e-05 2024-09-17T23:29:44.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483162+0000", 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8758999999999999e-05 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1074 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1318960550000003, 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037534672999999998, 2024-09-17T23:29:44.639 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1074 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445855+0000", 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6780999999999994e-05 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445858+0000", 2024-09-17T23:29:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6599999999999999e-06 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445864+0000", 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445869+0000", 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7400000000000004e-06 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483192+0000", 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037322704999999998 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483244+0000", 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2744999999999997e-05 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483293+0000", 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8792000000000003e-05 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1075 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1317789779999998, 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037543656000000002, 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446843+0000", 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096762300000000003 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446852+0000", 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9439999999999994e-06 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446861+0000", 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.541e-06 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446871+0000", 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0498e-05 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483324+0000", 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036452248999999999 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483375+0000", 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1780000000000002e-05 2024-09-17T23:29:44.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483419+0000", 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020999999999997e-05 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1083 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 860160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1305112470000003, 2024-09-17T23:29:44.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037242256000000001, 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1083 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447144+0000", 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.16e-07 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447151+0000", 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9450000000000004e-06 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447154+0000", 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8639999999999998e-06 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447159+0000", 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484301+0000", 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037141835999999998 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484347+0000", 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5998000000000002e-05 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484386+0000", 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9060000000000002e-05 2024-09-17T23:29:44.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1084 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 869376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1304905229999997, 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037333979000000003, 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1084 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.67e-07 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447166+0000", 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2500000000000001e-06 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447169+0000", 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3189999999999999e-06 2024-09-17T23:29:44.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447172+0000", 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484414+0000", 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037242311 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484458+0000", 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4515000000000003e-05 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484498+0000", 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9703000000000003e-05 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1085 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1304693759999997, 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037436900000000002, 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1085 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447300+0000", 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011436 2024-09-17T23:29:44.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447304+0000", 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2780000000000004e-06 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447312+0000", 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3440000000000001e-06 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447329+0000", 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6307000000000002e-05 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484526+0000", 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037196976999999999 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484581+0000", 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5019999999999998e-05 2024-09-17T23:29:44.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484622+0000", 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1613999999999998e-05 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1086 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 887808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1303077459999997, 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03739141, 2024-09-17T23:29:44.662 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1086 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447532+0000", 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018467099999999999 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447536+0000", 2024-09-17T23:29:44.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0899999999999998e-06 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447544+0000", 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2859999999999999e-06 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447565+0000", 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0706000000000001e-05 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484652+0000", 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037086898 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484696+0000", 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4159000000000002e-05 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484738+0000", 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-05 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1087 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 897024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1300766529999997, 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037273000000000001, 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1087 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447711+0000", 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013340800000000001 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447715+0000", 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9929999999999997e-06 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447724+0000", 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6009999999999995e-06 2024-09-17T23:29:44.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447730+0000", 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.852e-06 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484768+0000", 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037037887999999998 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484811+0000", 2024-09-17T23:29:44.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3736999999999998e-05 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484851+0000", 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9521000000000003e-05 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1088 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 906240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.1299045889999997, 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037212453999999999, 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1088 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447893+0000", 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000142993 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447901+0000", 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8930000000000005e-06 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447907+0000", 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3300000000000004e-06 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447911+0000", 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1910000000000002e-06 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484880+0000", 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036968207000000003 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484923+0000", 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3197e-05 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484962+0000", 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9643e-05 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:44.676 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:44.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:44.678 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:44.674+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:29:44.711 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:29:44.723 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-09-17T23:29:44.367665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484417896,"num_objects":409,"num_object_clones":134,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":163,"num_read":66381,"num_read_kb":106290,"num_write":3412,"num_write_kb":31968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":5183154,"num_keys_recovered":29,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3079,"ondisk_log_size":3079,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201684,"kb_used_data":10372,"kb_used_omap":9,"kb_used_meta":191158,"kb_avail":566029356,"statfs":{"total":579820584960,"available":579614060544,"internally_reserved":0,"allocated":10620928,"data_stored":39276380,"data_compressed":2385932,"data_compressed_allocated":4694016,"data_compressed_original":36576256,"omap_allocated":9566,"internal_metadata":195746466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":134612665,"num_objects":295,"num_object_clones":119,"num_object_copies":590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":295,"num_whiteouts":138,"num_read":44349,"num_read_kb":62883,"num_write":1928,"num_write_kb":17201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-5,"num_bytes_recovered":-47,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001041"},"pg_stats":[{"pgid":"3.1f","version":"42'135","reported_seq":733,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.310209+0000","last_change":"2024-09-17T23:29:16.849137+0000","last_active":"2024-09-17T23:29:43.310209+0000","last_peered":"2024-09-17T23:29:43.310209+0000","last_clean":"2024-09-17T23:29:43.310209+0000","last_became_active":"2024-09-17T23:29:15.605286+0000","last_became_peered":"2024-09-17T23:29:15.605286+0000","last_unstale":"2024-09-17T23:29:43.310209+0000","last_undegraded":"2024-09-17T23:29:43.310209+0000","last_fullsized":"2024-09-17T23:29:43.310209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:04.627499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055557299999999996,"stat_sum":{"num_bytes":24794164,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":992,"num_read_kb":1476,"num_write":182,"num_write_kb":653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"41'66","reported_seq":2769,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.968952+0000","last_change":"2024-09-17T23:29:27.994821+0000","last_active":"2024-09-17T23:29:42.968952+0000","last_peered":"2024-09-17T23:29:42.968952+0000","last_clean":"2024-09-17T23:29:42.968952+0000","last_became_active":"2024-09-17T23:29:27.894511+0000","last_became_peered":"2024-09-17T23:29:27.894511+0000","last_unstale":"2024-09-17T23:29:42.968952+0000","last_undegraded":"2024-09-17T23:29:42.968952+0000","last_fullsized":"2024-09-17T23:29:42.968952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:46.685028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055521100000000001,"stat_sum":{"num_bytes":71620169,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":2684,"num_read_kb":17496,"num_write":77,"num_write_kb":4683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":4447268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"41'59","reported_seq":1011,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993437+0000","last_change":"2024-09-17T23:29:33.928943+0000","last_active":"2024-09-17T23:29:42.993437+0000","last_peered":"2024-09-17T23:29:42.993437+0000","last_clean":"2024-09-17T23:29:42.993437+0000","last_became_active":"2024-09-17T23:29:33.928547+0000","last_became_peered":"2024-09-17T23:29:33.928547+0000","last_unstale":"2024-09-17T23:29:42.993437+0000","last_undegraded":"2024-09-17T23:29:42.993437+0000","last_fullsized":"2024-09-17T23:29:42.993437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:01:22.975622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476274,"stat_sum":{"num_bytes":12722688,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":1009,"num_read_kb":6158,"num_write":50,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":187904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"42'46","reported_seq":4478,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.339580+0000","last_change":"2024-09-17T23:29:39.959779+0000","last_active":"2024-09-17T23:29:43.339580+0000","last_peered":"2024-09-17T23:29:43.339580+0000","last_clean":"2024-09-17T23:29:43.339580+0000","last_became_active":"2024-09-17T23:29:39.959404+0000","last_became_peered":"2024-09-17T23:29:39.959404+0000","last_unstale":"2024-09-17T23:29:43.339580+0000","last_undegraded":"2024-09-17T23:29:43.339580+0000","last_fullsized":"2024-09-17T23:29:43.339580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:53:12.494960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263599,"stat_sum":{"num_bytes":1196032,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":4366,"num_read_kb":801,"num_write":37,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"42'65","reported_seq":11681,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.208401+0000","last_change":"2024-09-17T23:29:16.844175+0000","last_active":"2024-09-17T23:29:43.208401+0000","last_peered":"2024-09-17T23:29:43.208401+0000","last_clean":"2024-09-17T23:29:43.208401+0000","last_became_active":"2024-09-17T23:29:15.601606+0000","last_became_peered":"2024-09-17T23:29:15.601606+0000","last_unstale":"2024-09-17T23:29:43.208401+0000","last_undegraded":"2024-09-17T23:29:43.208401+0000","last_fullsized":"2024-09-17T23:29:43.208401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:14:25.640620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046809400000000001,"stat_sum":{"num_bytes":26266385,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":11641,"num_read_kb":12620,"num_write":80,"num_write_kb":804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"42'33","reported_seq":1422,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.342869+0000","last_change":"2024-09-17T23:29:39.959937+0000","last_active":"2024-09-17T23:29:43.342869+0000","last_peered":"2024-09-17T23:29:43.342869+0000","last_clean":"2024-09-17T23:29:43.342869+0000","last_became_active":"2024-09-17T23:29:39.959168+0000","last_became_peered":"2024-09-17T23:29:39.959168+0000","last_unstale":"2024-09-17T23:29:43.342869+0000","last_undegraded":"2024-09-17T23:29:43.342869+0000","last_fullsized":"2024-09-17T23:29:43.342869+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.775308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001041527,"stat_sum":{"num_bytes":21420067,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":1321,"num_read_kb":4647,"num_write":31,"num_write_kb":302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"41'144","reported_seq":1913,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.030198+0000","last_change":"2024-09-17T23:29:16.855904+0000","last_active":"2024-09-17T23:29:43.030198+0000","last_peered":"2024-09-17T23:29:43.030198+0000","last_clean":"2024-09-17T23:29:43.030198+0000","last_became_active":"2024-09-17T23:29:15.602040+0000","last_became_peered":"2024-09-17T23:29:15.602040+0000","last_unstale":"2024-09-17T23:29:43.030198+0000","last_undegraded":"2024-09-17T23:29:43.030198+0000","last_fullsized":"2024-09-17T23:29:43.030198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:17:11.968247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001306843,"stat_sum":{"num_bytes":12962816,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":2678,"num_read_kb":2749,"num_write":112,"num_write_kb":737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"41'78","reported_seq":681,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.968538+0000","last_change":"2024-09-17T23:29:16.844092+0000","last_active":"2024-09-17T23:29:42.968538+0000","last_peered":"2024-09-17T23:29:42.968538+0000","last_clean":"2024-09-17T23:29:42.968538+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:29:42.968538+0000","last_undegraded":"2024-09-17T23:29:42.968538+0000","last_fullsized":"2024-09-17T23:29:42.968538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:59.352646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045869699999999999,"stat_sum":{"num_bytes":7225856,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":1015,"num_read_kb":2072,"num_write":59,"num_write_kb":346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"39'116","reported_seq":1961,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.979717+0000","last_change":"2024-09-17T23:29:16.849195+0000","last_active":"2024-09-17T23:29:42.979717+0000","last_peered":"2024-09-17T23:29:42.979717+0000","last_clean":"2024-09-17T23:29:42.979717+0000","last_became_active":"2024-09-17T23:29:15.601179+0000","last_became_peered":"2024-09-17T23:29:15.601179+0000","last_unstale":"2024-09-17T23:29:42.979717+0000","last_undegraded":"2024-09-17T23:29:42.979717+0000","last_fullsized":"2024-09-17T23:29:42.979717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:11.439649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050274800000000002,"stat_sum":{"num_bytes":28191782,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2171,"num_read_kb":1781,"num_write":167,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"41'78","reported_seq":415,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.968695+0000","last_change":"2024-09-17T23:29:33.926843+0000","last_active":"2024-09-17T23:29:42.968695+0000","last_peered":"2024-09-17T23:29:42.968695+0000","last_clean":"2024-09-17T23:29:42.968695+0000","last_became_active":"2024-09-17T23:29:33.926463+0000","last_became_peered":"2024-09-17T23:29:33.926463+0000","last_unstale":"2024-09-17T23:29:42.968695+0000","last_undegraded":"2024-09-17T23:29:42.968695+0000","last_fullsized":"2024-09-17T23:29:42.968695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:04:02.852268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099808899999999996,"stat_sum":{"num_bytes":6405680,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1058,"num_read_kb":1863,"num_write":65,"num_write_kb":900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":42544,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"41'47","reported_seq":6214,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993720+0000","last_change":"2024-09-17T23:29:16.851584+0000","last_active":"2024-09-17T23:29:42.993720+0000","last_peered":"2024-09-17T23:29:42.993720+0000","last_clean":"2024-09-17T23:29:42.993720+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:29:42.993720+0000","last_undegraded":"2024-09-17T23:29:42.993720+0000","last_fullsized":"2024-09-17T23:29:42.993720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:01.976958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073596200000000005,"stat_sum":{"num_bytes":23856128,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":6116,"num_read_kb":6536,"num_write":46,"num_write_kb":480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"42'56","reported_seq":391,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.279185+0000","last_change":"2024-09-17T23:29:28.004020+0000","last_active":"2024-09-17T23:29:43.279185+0000","last_peered":"2024-09-17T23:29:43.279185+0000","last_clean":"2024-09-17T23:29:43.279185+0000","last_became_active":"2024-09-17T23:29:27.894409+0000","last_became_peered":"2024-09-17T23:29:27.894409+0000","last_unstale":"2024-09-17T23:29:43.279185+0000","last_undegraded":"2024-09-17T23:29:43.279185+0000","last_fullsized":"2024-09-17T23:29:43.279185+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:06.351708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013571150000000001,"stat_sum":{"num_bytes":237584,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":549,"num_read_kb":508,"num_write":47,"num_write_kb":318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":42496,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993682+0000","last_change":"2024-09-17T23:29:39.959940+0000","last_active":"2024-09-17T23:29:42.993682+0000","last_peered":"2024-09-17T23:29:42.993682+0000","last_clean":"2024-09-17T23:29:42.993682+0000","last_became_active":"2024-09-17T23:29:39.959465+0000","last_became_peered":"2024-09-17T23:29:39.959465+0000","last_unstale":"2024-09-17T23:29:42.993682+0000","last_undegraded":"2024-09-17T23:29:42.993682+0000","last_fullsized":"2024-09-17T23:29:42.993682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:35.867019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"41'384","reported_seq":506,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.030288+0000","last_change":"2024-09-17T23:29:16.856135+0000","last_active":"2024-09-17T23:29:43.030288+0000","last_peered":"2024-09-17T23:29:43.030288+0000","last_clean":"2024-09-17T23:29:43.030288+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:29:43.030288+0000","last_undegraded":"2024-09-17T23:29:43.030288+0000","last_fullsized":"2024-09-17T23:29:43.030288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:04.033088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001532662,"stat_sum":{"num_bytes":830551,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":1972,"num_read_kb":2254,"num_write":673,"num_write_kb":1199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.030290+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:29:43.030290+0000","last_peered":"2024-09-17T23:29:43.030290+0000","last_clean":"2024-09-17T23:29:43.030290+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:29:43.030290+0000","last_undegraded":"2024-09-17T23:29:43.030290+0000","last_fullsized":"2024-09-17T23:29:43.030290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"41'25","reported_seq":126,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.030287+0000","last_change":"2024-09-17T23:29:16.855828+0000","last_active":"2024-09-17T23:29:43.030287+0000","last_peered":"2024-09-17T23:29:43.030287+0000","last_clean":"2024-09-17T23:29:43.030287+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:29:43.030287+0000","last_undegraded":"2024-09-17T23:29:43.030287+0000","last_fullsized":"2024-09-17T23:29:43.030287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:51.709964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012515670000000001,"stat_sum":{"num_bytes":655360,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":40,"num_read_kb":447,"num_write":21,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.030205+0000","last_change":"2024-09-17T23:29:39.958959+0000","last_active":"2024-09-17T23:29:43.030205+0000","last_peered":"2024-09-17T23:29:43.030205+0000","last_clean":"2024-09-17T23:29:43.030205+0000","last_became_active":"2024-09-17T23:29:39.958620+0000","last_became_peered":"2024-09-17T23:29:39.958620+0000","last_unstale":"2024-09-17T23:29:43.030205+0000","last_undegraded":"2024-09-17T23:29:43.030205+0000","last_fullsized":"2024-09-17T23:29:43.030205+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:19.215313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"41'59","reported_seq":299,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.030264+0000","last_change":"2024-09-17T23:29:16.855577+0000","last_active":"2024-09-17T23:29:43.030264+0000","last_peered":"2024-09-17T23:29:43.030264+0000","last_clean":"2024-09-17T23:29:43.030264+0000","last_became_active":"2024-09-17T23:29:15.600130+0000","last_became_peered":"2024-09-17T23:29:15.600130+0000","last_unstale":"2024-09-17T23:29:43.030264+0000","last_undegraded":"2024-09-17T23:29:43.030264+0000","last_fullsized":"2024-09-17T23:29:43.030264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:08.877012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010554410000000001,"stat_sum":{"num_bytes":5297152,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":183,"num_read_kb":337,"num_write":54,"num_write_kb":503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.979573+0000","last_change":"2024-09-17T23:29:08.767780+0000","last_active":"2024-09-17T23:29:42.979573+0000","last_peered":"2024-09-17T23:29:42.979573+0000","last_clean":"2024-09-17T23:29:42.979573+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:29:42.979573+0000","last_undegraded":"2024-09-17T23:29:42.979573+0000","last_fullsized":"2024-09-17T23:29:42.979573+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:47:22.262418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"41'38","reported_seq":5622,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.968752+0000","last_change":"2024-09-17T23:29:33.928884+0000","last_active":"2024-09-17T23:29:42.968752+0000","last_peered":"2024-09-17T23:29:42.968752+0000","last_clean":"2024-09-17T23:29:42.968752+0000","last_became_active":"2024-09-17T23:29:33.928630+0000","last_became_peered":"2024-09-17T23:29:33.928630+0000","last_unstale":"2024-09-17T23:29:42.968752+0000","last_undegraded":"2024-09-17T23:29:42.968752+0000","last_fullsized":"2024-09-17T23:29:42.968752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:51.963261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14785568,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":5570,"num_read_kb":8799,"num_write":41,"num_write_kb":1199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"41'27","reported_seq":309,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993480+0000","last_change":"2024-09-17T23:29:27.969780+0000","last_active":"2024-09-17T23:29:42.993480+0000","last_peered":"2024-09-17T23:29:42.993480+0000","last_clean":"2024-09-17T23:29:42.993480+0000","last_became_active":"2024-09-17T23:29:27.896108+0000","last_became_peered":"2024-09-17T23:29:27.896108+0000","last_unstale":"2024-09-17T23:29:42.993480+0000","last_undegraded":"2024-09-17T23:29:42.993480+0000","last_fullsized":"2024-09-17T23:29:42.993480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:59.726969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000582823,"stat_sum":{"num_bytes":9375744,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":188,"num_read_kb":371,"num_write":22,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":219648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":69,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993322+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:29:42.993322+0000","last_peered":"2024-09-17T23:29:42.993322+0000","last_clean":"2024-09-17T23:29:42.993322+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:29:42.993322+0000","last_undegraded":"2024-09-17T23:29:42.993322+0000","last_fullsized":"2024-09-17T23:29:42.993322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"42'139","reported_seq":908,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.337073+0000","last_change":"2024-09-17T23:29:27.926562+0000","last_active":"2024-09-17T23:29:43.337073+0000","last_peered":"2024-09-17T23:29:43.337073+0000","last_clean":"2024-09-17T23:29:43.337073+0000","last_became_active":"2024-09-17T23:29:27.893418+0000","last_became_peered":"2024-09-17T23:29:27.893418+0000","last_unstale":"2024-09-17T23:29:43.337073+0000","last_undegraded":"2024-09-17T23:29:43.337073+0000","last_fullsized":"2024-09-17T23:29:43.337073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:23:32.217635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363521,"stat_sum":{"num_bytes":1075262,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1795,"num_read_kb":1687,"num_write":155,"num_write_kb":366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.968621+0000","last_change":"2024-09-17T23:29:09.377458+0000","last_active":"2024-09-17T23:29:42.968621+0000","last_peered":"2024-09-17T23:29:42.968621+0000","last_clean":"2024-09-17T23:29:42.968621+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:29:42.968621+0000","last_undegraded":"2024-09-17T23:29:42.968621+0000","last_fullsized":"2024-09-17T23:29:42.968621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:59.106784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"41'205","reported_seq":2489,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.305584+0000","last_change":"2024-09-17T23:29:30.917501+0000","last_active":"2024-09-17T23:29:43.305584+0000","last_peered":"2024-09-17T23:29:43.305584+0000","last_clean":"2024-09-17T23:29:43.305584+0000","last_became_active":"2024-09-17T23:29:30.917041+0000","last_became_peered":"2024-09-17T23:29:30.917041+0000","last_unstale":"2024-09-17T23:29:43.305584+0000","last_undegraded":"2024-09-17T23:29:43.305584+0000","last_fullsized":"2024-09-17T23:29:43.305584+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:52:40.340295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00105678,"stat_sum":{"num_bytes":25872422,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":3623,"num_read_kb":4679,"num_write":135,"num_write_kb":1025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":86528,"num_keys_recovered":17,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":38,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.979670+0000","last_change":"2024-09-17T23:29:33.927055+0000","last_active":"2024-09-17T23:29:42.979670+0000","last_peered":"2024-09-17T23:29:42.979670+0000","last_clean":"2024-09-17T23:29:42.979670+0000","last_became_active":"2024-09-17T23:29:33.926771+0000","last_became_peered":"2024-09-17T23:29:33.926771+0000","last_unstale":"2024-09-17T23:29:42.979670+0000","last_undegraded":"2024-09-17T23:29:42.979670+0000","last_fullsized":"2024-09-17T23:29:42.979670+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:01.969338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"42'118","reported_seq":262,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.154759+0000","last_change":"2024-09-17T23:29:27.947652+0000","last_active":"2024-09-17T23:29:43.154759+0000","last_peered":"2024-09-17T23:29:43.154759+0000","last_clean":"2024-09-17T23:29:43.154759+0000","last_became_active":"2024-09-17T23:29:27.894483+0000","last_became_peered":"2024-09-17T23:29:27.894483+0000","last_unstale":"2024-09-17T23:29:43.154759+0000","last_undegraded":"2024-09-17T23:29:43.154759+0000","last_fullsized":"2024-09-17T23:29:43.154759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:12.598756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047998399999999998,"stat_sum":{"num_bytes":11660369,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":10,"num_read":417,"num_read_kb":1008,"num_write":161,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":63,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.971939+0000","last_change":"2024-09-17T23:29:39.959552+0000","last_active":"2024-09-17T23:29:42.971939+0000","last_peered":"2024-09-17T23:29:42.971939+0000","last_clean":"2024-09-17T23:29:42.971939+0000","last_became_active":"2024-09-17T23:29:39.959224+0000","last_became_peered":"2024-09-17T23:29:39.959224+0000","last_unstale":"2024-09-17T23:29:42.971939+0000","last_undegraded":"2024-09-17T23:29:42.971939+0000","last_fullsized":"2024-09-17T23:29:42.971939+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:35.806577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":172,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.979774+0000","last_change":"2024-09-17T23:29:39.959742+0000","last_active":"2024-09-17T23:29:42.979774+0000","last_peered":"2024-09-17T23:29:42.979774+0000","last_clean":"2024-09-17T23:29:42.979774+0000","last_became_active":"2024-09-17T23:29:39.959373+0000","last_became_peered":"2024-09-17T23:29:39.959373+0000","last_unstale":"2024-09-17T23:29:42.979774+0000","last_undegraded":"2024-09-17T23:29:42.979774+0000","last_fullsized":"2024-09-17T23:29:42.979774+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:43:46.897381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"41'153","reported_seq":6611,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.344113+0000","last_change":"2024-09-17T23:29:27.961774+0000","last_active":"2024-09-17T23:29:43.344113+0000","last_peered":"2024-09-17T23:29:43.344113+0000","last_clean":"2024-09-17T23:29:43.344113+0000","last_became_active":"2024-09-17T23:29:27.893544+0000","last_became_peered":"2024-09-17T23:29:27.893544+0000","last_unstale":"2024-09-17T23:29:43.344113+0000","last_undegraded":"2024-09-17T23:29:43.344113+0000","last_fullsized":"2024-09-17T23:29:43.344113+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:30.168960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001536372,"stat_sum":{"num_bytes":14198361,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":7304,"num_read_kb":7764,"num_write":110,"num_write_kb":824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46096,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"42'105","reported_seq":2472,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.331717+0000","last_change":"2024-09-17T23:29:16.850836+0000","last_active":"2024-09-17T23:29:43.331717+0000","last_peered":"2024-09-17T23:29:43.331717+0000","last_clean":"2024-09-17T23:29:43.331717+0000","last_became_active":"2024-09-17T23:29:15.602469+0000","last_became_peered":"2024-09-17T23:29:15.602469+0000","last_unstale":"2024-09-17T23:29:43.331717+0000","last_undegraded":"2024-09-17T23:29:43.331717+0000","last_fullsized":"2024-09-17T23:29:43.331717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:50:56.408114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483418,"stat_sum":{"num_bytes":1507365,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":1228,"num_read_kb":2387,"num_write":64,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"42'57","reported_seq":361,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993448+0000","last_change":"2024-09-17T23:29:16.851336+0000","last_active":"2024-09-17T23:29:42.993448+0000","last_peered":"2024-09-17T23:29:42.993448+0000","last_clean":"2024-09-17T23:29:42.993448+0000","last_became_active":"2024-09-17T23:29:15.605424+0000","last_became_peered":"2024-09-17T23:29:15.605424+0000","last_unstale":"2024-09-17T23:29:42.993448+0000","last_undegraded":"2024-09-17T23:29:42.993448+0000","last_fullsized":"2024-09-17T23:29:42.993448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:16.050224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060532200000000004,"stat_sum":{"num_bytes":3486208,"num_objects":11,"num_object_clones":5,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":264,"num_read_kb":1526,"num_write":53,"num_write_kb":1002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"42'76","reported_seq":1373,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.332290+0000","last_change":"2024-09-17T23:29:16.844405+0000","last_active":"2024-09-17T23:29:43.332290+0000","last_peered":"2024-09-17T23:29:43.332290+0000","last_clean":"2024-09-17T23:29:43.332290+0000","last_became_active":"2024-09-17T23:29:15.601213+0000","last_became_peered":"2024-09-17T23:29:15.601213+0000","last_unstale":"2024-09-17T23:29:43.332290+0000","last_undegraded":"2024-09-17T23:29:43.332290+0000","last_fullsized":"2024-09-17T23:29:43.332290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:59:04.495378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049268800000000002,"stat_sum":{"num_bytes":19552784,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":1771,"num_read_kb":1298,"num_write":53,"num_write_kb":560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"42'111","reported_seq":1005,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.047691+0000","last_change":"2024-09-17T23:29:16.851583+0000","last_active":"2024-09-17T23:29:43.047691+0000","last_peered":"2024-09-17T23:29:43.047691+0000","last_clean":"2024-09-17T23:29:43.047691+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:29:43.047691+0000","last_undegraded":"2024-09-17T23:29:43.047691+0000","last_fullsized":"2024-09-17T23:29:43.047691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:33.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187900000000002,"stat_sum":{"num_bytes":2671675,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1782,"num_read_kb":1625,"num_write":71,"num_write_kb":1212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"39'49","reported_seq":499,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.975733+0000","last_change":"2024-09-17T23:29:16.847782+0000","last_active":"2024-09-17T23:29:42.975733+0000","last_peered":"2024-09-17T23:29:42.975733+0000","last_clean":"2024-09-17T23:29:42.975733+0000","last_became_active":"2024-09-17T23:29:15.599701+0000","last_became_peered":"2024-09-17T23:29:15.599701+0000","last_unstale":"2024-09-17T23:29:42.975733+0000","last_undegraded":"2024-09-17T23:29:42.975733+0000","last_fullsized":"2024-09-17T23:29:42.975733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:39:35.286867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095122,"stat_sum":{"num_bytes":17910340,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":443,"num_read_kb":549,"num_write":52,"num_write_kb":574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"42'30","reported_seq":120,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.089961+0000","last_change":"2024-09-17T23:29:33.928677+0000","last_active":"2024-09-17T23:29:43.089961+0000","last_peered":"2024-09-17T23:29:43.089961+0000","last_clean":"2024-09-17T23:29:43.089961+0000","last_became_active":"2024-09-17T23:29:33.928254+0000","last_became_peered":"2024-09-17T23:29:33.928254+0000","last_unstale":"2024-09-17T23:29:43.089961+0000","last_undegraded":"2024-09-17T23:29:43.089961+0000","last_fullsized":"2024-09-17T23:29:43.089961+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:46.295291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093123200000000004,"stat_sum":{"num_bytes":8448085,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":48,"num_read_kb":81,"num_write":30,"num_write_kb":1273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"42'62","reported_seq":565,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.343345+0000","last_change":"2024-09-17T23:29:16.851833+0000","last_active":"2024-09-17T23:29:43.343345+0000","last_peered":"2024-09-17T23:29:43.343345+0000","last_clean":"2024-09-17T23:29:43.343345+0000","last_became_active":"2024-09-17T23:29:15.606245+0000","last_became_peered":"2024-09-17T23:29:15.606245+0000","last_unstale":"2024-09-17T23:29:43.343345+0000","last_undegraded":"2024-09-17T23:29:43.343345+0000","last_fullsized":"2024-09-17T23:29:43.343345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:41:36.770658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078560200000000005,"stat_sum":{"num_bytes":3995188,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":539,"num_read_kb":3276,"num_write":70,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'35","reported_seq":126,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.307480+0000","last_change":"2024-09-17T23:29:39.959591+0000","last_active":"2024-09-17T23:29:43.307480+0000","last_peered":"2024-09-17T23:29:43.307480+0000","last_clean":"2024-09-17T23:29:43.307480+0000","last_became_active":"2024-09-17T23:29:39.959278+0000","last_became_peered":"2024-09-17T23:29:39.959278+0000","last_unstale":"2024-09-17T23:29:43.307480+0000","last_undegraded":"2024-09-17T23:29:43.307480+0000","last_fullsized":"2024-09-17T23:29:43.307480+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:35.146752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056375800000000003,"stat_sum":{"num_bytes":788518,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":30,"num_read_kb":316,"num_write":31,"num_write_kb":518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"42'62","reported_seq":123,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.971961+0000","last_change":"2024-09-17T23:29:39.959636+0000","last_active":"2024-09-17T23:29:42.971961+0000","last_peered":"2024-09-17T23:29:42.971961+0000","last_clean":"2024-09-17T23:29:42.971961+0000","last_became_active":"2024-09-17T23:29:39.959197+0000","last_became_peered":"2024-09-17T23:29:39.959197+0000","last_unstale":"2024-09-17T23:29:42.971961+0000","last_undegraded":"2024-09-17T23:29:42.971961+0000","last_fullsized":"2024-09-17T23:29:42.971961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:23.768273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043017799999999997,"stat_sum":{"num_bytes":32563200,"num_objects":17,"num_object_clones":8,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":849,"num_read_kb":1773,"num_write":55,"num_write_kb":1262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":110592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"42'70","reported_seq":1323,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:43.347549+0000","last_change":"2024-09-17T23:29:16.848895+0000","last_active":"2024-09-17T23:29:43.347549+0000","last_peered":"2024-09-17T23:29:43.347549+0000","last_clean":"2024-09-17T23:29:43.347549+0000","last_became_active":"2024-09-17T23:29:15.601164+0000","last_became_peered":"2024-09-17T23:29:15.601164+0000","last_unstale":"2024-09-17T23:29:43.347549+0000","last_undegraded":"2024-09-17T23:29:43.347549+0000","last_fullsized":"2024-09-17T23:29:43.347549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:33:17.940920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034516700000000001,"stat_sum":{"num_bytes":58248192,"num_objects":17,"num_object_clones":7,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":9,"num_read":1224,"num_read_kb":5344,"num_write":68,"num_write_kb":2098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"41'231","reported_seq":661,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-17T23:29:42.993686+0000","last_change":"2024-09-17T23:29:39.959921+0000","last_active":"2024-09-17T23:29:42.993686+0000","last_peered":"2024-09-17T23:29:42.993686+0000","last_clean":"2024-09-17T23:29:42.993686+0000","last_became_active":"2024-09-17T23:29:39.959472+0000","last_became_peered":"2024-09-17T23:29:39.959472+0000","last_unstale":"2024-09-17T23:29:42.993686+0000","last_undegraded":"2024-09-17T23:29:42.993686+0000","last_fullsized":"2024-09-17T23:29:42.993686+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:45:07.061338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001102308,"stat_sum":{"num_bytes":14005814,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1395,"num_read_kb":1962,"num_write":379,"num_write_kb":910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":483827509,"num_objects":405,"num_object_clones":134,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":163,"num_read":66265,"num_read_kb":106190,"num_write":3251,"num_write_kb":29550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":5183154,"num_keys_recovered":29,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7766016,"data_stored":37029482,"data_compressed":2376048,"data_compressed_allocated":4620288,"data_compressed_original":35429376,"omap_allocated":0,"internal_metadata":0},"log_size":2959,"ondisk_log_size":2959,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640268,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1228800,"data_stored":3554338,"data_compressed":245993,"data_compressed_allocated":466944,"data_compressed_original":3224576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640268,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1171456,"data_stored":2546752,"data_compressed":119089,"data_compressed_allocated":299008,"data_compressed_original":2160128,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640268,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37532,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334308,"statfs":{"total":96636764160,"available":96598331392,"internally_reserved":0,"allocated":2220032,"data_stored":10044902,"data_compressed":484388,"data_compressed_allocated":1069056,"data_compressed_original":9501184,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640268,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33492,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338348,"statfs":{"total":96636764160,"available":96602468352,"internally_reserved":0,"allocated":2342912,"data_stored":9613675,"data_compressed":588061,"data_compressed_allocated":1171456,"data_compressed_original":9076736,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640269,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1396736,"data_stored":4795705,"data_compressed":310570,"data_compressed_allocated":557056,"data_compressed_original":4425728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2260992,"data_stored":8721008,"data_compressed":637831,"data_compressed_allocated":1130496,"data_compressed_original":8187904,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1794048,"data_stored":8539818,"data_compressed":637831,"data_compressed_allocated":1130496,"data_compressed_original":8187904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":876544,"data_stored":4024123,"data_compressed":305628,"data_compressed_allocated":520192,"data_compressed_original":3852288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":9437708,"data_compressed":588061,"data_compressed_allocated":1171456,"data_compressed_original":9076736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1769472,"data_stored":9868994,"data_compressed":484388,"data_compressed_allocated":1069056,"data_compressed_original":9501184,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":2370774,"data_compressed":119089,"data_compressed_allocated":299008,"data_compressed_original":2160128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":2788065,"data_compressed":241051,"data_compressed_allocated":430080,"data_compressed_original":2651136,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:29:44.726 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [5, 2, 1, 4] 2024-09-17T23:29:44.726 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.15', '5', '2', '1', '4'] 2024-09-17T23:29:44.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.15 5 2 1 4 2024-09-17T23:29:44.778 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:44.776+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:44.819 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:44.819 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:44.819 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:44.819 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:44.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:44.878 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:44.876+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:44.937 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:44.937 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:44.937 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:44.937 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:44.937 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:44.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:44.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:29:44.990 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:44.988+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:45.064 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14717 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.026650311, 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007005636, 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14717 2024-09-17T23:29:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034530+0000", 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.624e-06 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034535+0000", 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0170000000000004e-06 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034553+0000", 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034567+0000", 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4446e-05 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034630+0000", 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2283e-05 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034770+0000", 2024-09-17T23:29:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014049000000000001 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041534+0000", 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067642350000000004 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14740 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0265915989999996, 2024-09-17T23:29:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070617650000000002, 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14740 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9999999999999997e-07 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034592+0000", 2024-09-17T23:29:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4010000000000001e-06 2024-09-17T23:29:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034602+0000", 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0638e-05 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034608+0000", 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.241e-06 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035105+0000", 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049700000000000005 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035244+0000", 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013833000000000001 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041649+0000", 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064055550000000003 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14760 3.11 3:8ddf073a:::rbd_data.1152c1142907.0000000000000029:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.026557188, 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010811016999999999, 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14760 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034623+0000", 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034627+0000", 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4680000000000003e-06 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034635+0000", 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034641+0000", 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:29:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035590+0000", 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094941100000000005 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036421+0000", 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083071299999999996 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036612+0000", 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191342 2024-09-17T23:29:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041698+0000", 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050861980000000001 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045354+0000", 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036561440000000001 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045394+0000", 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9604999999999998e-05 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045433+0000", 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8945000000000001e-05 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0255232530000002, 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:29:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0253503129999997, 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:29:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0253104430000004, 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:29:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:29:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14758 3.1 e41/33 3:853c0e15:::rbd_data.1152c1142907.0000000000000028:head v 41'194, mlcod=39'189)", 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0239460730000003, 2024-09-17T23:29:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010716488999999999, 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037234+0000", 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5000000000000002e-07 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037239+0000", 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.784e-06 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037249+0000", 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8889999999999997e-06 2024-09-17T23:29:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037257+0000", 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.379e-06 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038255+0000", 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099873299999999991 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038288+0000", 2024-09-17T23:29:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292000000000002e-05 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047928+0000", 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096399710000000007 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047950+0000", 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1790999999999999e-05 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14752 3.5 e41/36 3:a0bd3f31:::rbd_data.1152c1142907.0000000000000025:head v 41'34, mlcod=39'31)", 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0220920749999998, 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081097770000000003, 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039096+0000", 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2230000000000007e-06 2024-09-17T23:29:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039103+0000", 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7370000000000006e-06 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039109+0000", 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039160+0000", 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1072999999999997e-05 2024-09-17T23:29:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039178+0000", 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8771e-05 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047154+0000", 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079759010000000005 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047197+0000", 2024-09-17T23:29:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2457999999999999e-05 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14838 3.3 e41/30 3:c39663f8:::rbd_data.1152c1142907.0000000000000064:head v 41'27, mlcod=39'22)", 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0184386129999998, 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071267700000000002, 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5300000000000003e-07 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042747+0000", 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042759+0000", 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2138e-05 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042766+0000", 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979999999999997e-06 2024-09-17T23:29:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042818+0000", 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2191000000000002e-05 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042840+0000", 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1535e-05 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049823+0000", 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069831559999999999 2024-09-17T23:29:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049867+0000", 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.439e-05 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14782 3.1 e41/33 3:801bb480:::rbd_data.1152c1142907.0000000000000034:head v 41'196, mlcod=39'189)", 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.043221+0000", 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.017957709, 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066679419999999996, 2024-09-17T23:29:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043221+0000", 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043221+0000", 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043222+0000", 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4899999999999995e-07 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043235+0000", 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2875e-05 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043244+0000", 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5559999999999996e-06 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043251+0000", 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6930000000000001e-06 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043299+0000", 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7525000000000003e-05 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.043320+0000", 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1208000000000001e-05 2024-09-17T23:29:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049851+0000", 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065308900000000001 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049889+0000", 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8546e-05 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14800 3.1 e41/33 3:8381b477:::rbd_data.1152c1142907.000000000000003d:head v 41'198, mlcod=39'189)", 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0162382560000003, 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007659412, 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044942+0000", 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-07 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044947+0000", 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.789e-06 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044958+0000", 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0223e-05 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044966+0000", 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999994e-06 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045312+0000", 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034605399999999999 2024-09-17T23:29:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045344+0000", 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1999999999999999e-05 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052554+0000", 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072101120000000003 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052600+0000", 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6521999999999999e-05 2024-09-17T23:29:45.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21686 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7339994399999998, 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010211289, 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21686 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327182+0000", 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.199e-06 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327654+0000", 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000472094 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327668+0000", 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3867e-05 2024-09-17T23:29:45.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327676+0000", 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9289999999999998e-06 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327897+0000", 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022101 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327979+0000", 2024-09-17T23:29:45.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1921000000000004e-05 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328181+0000", 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020200799999999999 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337270+0000", 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090895820000000006 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337286+0000", 2024-09-17T23:29:45.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6112000000000001e-05 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337344+0000", 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7989999999999999e-05 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337391+0000", 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6576999999999998e-05 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21687 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7334782989999997, 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098426639999999992, 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21687 2024-09-17T23:29:45.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5700000000000002e-07 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327704+0000", 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9349999999999999e-06 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327713+0000", 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-06 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327719+0000", 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9059999999999999e-06 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328700+0000", 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00098110899999999997 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328774+0000", 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3796999999999998e-05 2024-09-17T23:29:45.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328937+0000", 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162934 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337441+0000", 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085044609999999996 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337457+0000", 2024-09-17T23:29:45.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6378000000000001e-05 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337503+0000", 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5546000000000001e-05 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337543+0000", 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0559999999999998e-05 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21676 3.1 e41/33 3:87de3167:::rbd_data.1152c1142907.0000000000000151:head v 41'200, mlcod=41'196)", 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7334386589999999, 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097022959999999991, 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327741+0000", 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327746+0000", 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-17T23:29:45.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327754+0000", 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999992e-06 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327760+0000", 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327819+0000", 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9691000000000002e-05 2024-09-17T23:29:45.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327838+0000", 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8691000000000001e-05 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337396+0000", 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095580470000000001 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337443+0000", 2024-09-17T23:29:45.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6629999999999999e-05 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21673 3.8 e41/30 3:10761a55:::rbd_data.1152c1142907.0000000000000150:head v 41'150, mlcod=41'146)", 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327956+0000", 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7332234619999998, 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069529709999999996, 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327956+0000", 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327956+0000", 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327956+0000", 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:29:45.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327971+0000", 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5075000000000001e-05 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327981+0000", 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0121e-05 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327988+0000", 2024-09-17T23:29:45.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328057+0000", 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8768000000000003e-05 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328075+0000", 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8674000000000001e-05 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334865+0000", 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067897010000000004 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334909+0000", 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3566000000000003e-05 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21694 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:45.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7324891220000005, 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090673060000000007, 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21694 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328691+0000", 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328977+0000", 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028580700000000001 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328995+0000", 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329004+0000", 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3410000000000003e-06 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330527+0000", 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001522851 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330615+0000", 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8366000000000001e-05 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330857+0000", 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024195999999999999 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337600+0000", 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067435300000000002 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337662+0000", 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1246000000000003e-05 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337721+0000", 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8860000000000002e-05 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337757+0000", 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6749999999999999e-05 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21695 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7319869570000002, 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010154599, 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21695 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329193+0000", 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4499999999999995e-07 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329198+0000", 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329213+0000", 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4606999999999999e-05 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329220+0000", 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.328e-06 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331593+0000", 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002373287 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331670+0000", 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6156999999999996e-05 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331868+0000", 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019856600000000001 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337649+0000", 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057810889999999997 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339251+0000", 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016017309999999999 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339305+0000", 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3594000000000002e-05 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339347+0000", 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999997e-05 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21693 3.8 e41/30 3:12e31d2e:::rbd_data.1152c1142907.000000000000001a:head v 41'153, mlcod=41'146)", 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7315711770000002, 2024-09-17T23:29:45.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072668539999999997, 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:45.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329609+0000", 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8899999999999999e-07 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329614+0000", 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5639999999999996e-06 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329623+0000", 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329633+0000", 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0003000000000001e-05 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331519+0000", 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018852509999999999 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331536+0000", 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7393000000000001e-05 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336822+0000", 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052855519999999998 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336875+0000", 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3137999999999998e-05 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7308062159999995, 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:45.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:29:45.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:29:45.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21704 3.0 e41/30 3:07ab2e9c:::rbd_data.1152c1142907.0000000000000154:head v 41'130, mlcod=41'127)", 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.333193+0000", 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7279865619999999, 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0061139330000000002, 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333193+0000", 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333193+0000", 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333194+0000", 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8299999999999995e-07 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333202+0000", 2024-09-17T23:29:45.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5280000000000003e-06 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333214+0000", 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2332e-05 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333221+0000", 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8589999999999999e-06 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:45.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333272+0000", 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0856000000000002e-05 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.333294+0000", 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.175e-05 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339262+0000", 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059679570000000003 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:45.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339306+0000", 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4668e-05 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:45.166 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:45.167 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone11 from 0x96ff600 (overlap 0x1c34000) to 0x2d2000 2024-09-17T23:29:45.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:45.168 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:45.164+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:45.268 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:45.265+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:45.281 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:45.282 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:45.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:45.368 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:45.367+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:45.400 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:45.400 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:45.400 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:45.400 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:45.400 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:45.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:45.456 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T23:29:45.493 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:45.491+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:45.512 INFO:teuthology.orchestra.run.smithi139.stdout:1235 write 0x185bc00 thru 0x18619d8 (0x5dd9 bytes) 2024-09-17T23:29:45.521 INFO:teuthology.orchestra.run.smithi139.stdout:1236 trunc from 0x96ff600 to 0x311ce00 2024-09-17T23:29:45.568 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.690756882999999, 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.569 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:29:45.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.675131752999999, 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:29:45.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:29:45.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.675094096999999, 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.581 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:29:45.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.675076665999999, 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:29:45.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:29:45.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.675041735000001, 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.592 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:29:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:29:45.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.675010315000002, 2024-09-17T23:29:45.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:29:45.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:29:45.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.674993689000001, 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:29:45.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:29:45.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:29:45.607 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.674971995, 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.608 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:29:45.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.674960277, 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:29:45.613 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:29:45.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3221 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.618 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454720+0000", 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.110782843, 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045766706999999997, 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3221 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454720+0000", 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454720+0000", 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454721+0000", 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.89e-07 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454723+0000", 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9860000000000001e-06 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454732+0000", 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8859999999999993e-06 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454737+0000", 2024-09-17T23:29:45.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3879999999999999e-06 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500394+0000", 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045657375 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500437+0000", 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2165e-05 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500487+0000", 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0518e-05 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.1107579340000004, 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:29:45.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:29:45.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.627 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.1107354679999997, 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.628 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:29:45.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3239 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 355328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.458109+0000", 2024-09-17T23:29:45.632 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.1073944649999996, 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045774201, 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3239 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.458109+0000", 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.458109+0000", 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.459817+0000", 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017079700000000001 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.459826+0000", 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1630000000000002e-06 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.459836+0000", 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0609e-05 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.459843+0000", 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6209999999999998e-06 2024-09-17T23:29:45.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.503770+0000", 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043927364000000003 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.503827+0000", 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6301000000000002e-05 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.503883+0000", 2024-09-17T23:29:45.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6172999999999999e-05 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3250 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 410624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.460001+0000", 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.1055026459999997, 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045751235000000001, 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.637 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3250 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.460001+0000", 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.460001+0000", 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.461816+0000", 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001815042 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.461820+0000", 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0459999999999999e-06 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.461828+0000", 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6610000000000006e-06 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.461833+0000", 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4769999999999997e-06 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.505638+0000", 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043805368999999997 2024-09-17T23:29:45.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.505690+0000", 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2058999999999997e-05 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.505752+0000", 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1581000000000006e-05 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3262 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.1032906889999996, 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045783590999999998, 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.643 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3262 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463817+0000", 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00160411 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463821+0000", 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6670000000000004e-06 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463838+0000", 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6898999999999999e-05 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463848+0000", 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507787+0000", 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043939260000000001 2024-09-17T23:29:45.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507895+0000", 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010859899999999999 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507996+0000", 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100587 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3317 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 760832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472096+0000", 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.0934069869999998, 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045763674999999997, 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3317 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.648 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472096+0000", 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472096+0000", 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472096+0000", 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1e-07 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472097+0000", 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.06e-06 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472100+0000", 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0419999999999998e-06 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472103+0000", 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5490000000000001e-06 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517617+0000", 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045514116 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517727+0000", 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000110092 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517860+0000", 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013260600000000001 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.652 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.0933971839999996, 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:29:45.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.0914892900000002, 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:29:45.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:45.661 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.0895042200000002, 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:29:45.665 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.0894946250000004, 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:45.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:29:45.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:45.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:45.673 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:45.673 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:45.670+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:45.674 INFO:teuthology.orchestra.run.smithi139.stdout:1237 punch from 0x2bc4c00 to 0x2bcb600, (0x6a00 bytes) 2024-09-17T23:29:45.674 INFO:teuthology.orchestra.run.smithi139.stdout:1238 trunc from 0x311ce00 to 0xdd14800 2024-09-17T23:29:45.674 INFO:teuthology.orchestra.run.smithi139.stdout:1239 read 0x6e1a600 thru 0x6e24440 (0x9e41 bytes) 2024-09-17T23:29:45.674 INFO:teuthology.orchestra.run.smithi139.stdout:1241 read 0x372ae00 thru 0x372b8b0 (0xab1 bytes) 2024-09-17T23:29:45.674 INFO:teuthology.orchestra.run.smithi139.stdout:1242 punch from 0x4ac2200 to 0x4ace800, (0xc600 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1246 read 0x1c19c00 thru 0x1c232fb (0x96fc bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1248 write 0xc39e400 thru 0xc39f1a9 (0xdaa bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1249 write 0x4e39c00 thru 0x4e44c61 (0xb062 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1250 trunc from 0xdd14800 to 0xb84bc00 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1251 write 0x5ca0e00 thru 0x5ca9c5f (0x8e60 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1253 write 0x51a2e00 thru 0x51a90d0 (0x62d1 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1255 read 0xb1eb600 thru 0xb1f859f (0xcfa0 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1258 write 0x971ec00 thru 0x9724b72 (0x5f73 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1259 write 0x500d800 thru 0x50136ad (0x5eae bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1260 punch from 0x6064c00 to 0x6066800, (0x1c00 bytes) 2024-09-17T23:29:45.675 INFO:teuthology.orchestra.run.smithi139.stdout:1263 read 0x299ec00 thru 0x29a6c48 (0x8049 bytes) 2024-09-17T23:29:45.676 INFO:teuthology.orchestra.run.smithi139.stdout:1266 punch from 0x6cdfe00 to 0x6ce5e00, (0x6000 bytes) 2024-09-17T23:29:45.676 INFO:teuthology.orchestra.run.smithi139.stdout:1267 read 0x47e1c00 thru 0x47e3980 (0x1d81 bytes) 2024-09-17T23:29:45.676 INFO:teuthology.orchestra.run.smithi139.stdout:1271 read 0x8c79600 thru 0x8c86e10 (0xd811 bytes) 2024-09-17T23:29:45.676 INFO:teuthology.orchestra.run.smithi139.stdout:1273 write 0x238c800 thru 0x2399853 (0xd054 bytes) 2024-09-17T23:29:45.676 INFO:teuthology.orchestra.run.smithi139.stdout:1274 write 0x2142c00 thru 0x214351b (0x91c bytes) 2024-09-17T23:29:45.677 INFO:teuthology.orchestra.run.smithi139.stdout:1275 trunc from 0xb84bc00 to 0xb0f3600 2024-09-17T23:29:45.698 INFO:teuthology.orchestra.run.smithi139.stdout:1276 write 0xae4be00 thru 0xae50ecc (0x50cd bytes) 2024-09-17T23:29:45.699 INFO:teuthology.orchestra.run.smithi139.stdout:1280 read 0x894c800 thru 0x895c4a3 (0xfca4 bytes) 2024-09-17T23:29:45.719 INFO:teuthology.orchestra.run.smithi139.stdout:1281 write 0xb816000 thru 0xb816967 (0x968 bytes) 2024-09-17T23:29:45.719 INFO:teuthology.orchestra.run.smithi139.stdout:1283 write 0x1991e00 thru 0x19a02c4 (0xe4c5 bytes) 2024-09-17T23:29:45.719 INFO:teuthology.orchestra.run.smithi139.stdout:1284 read 0xd18800 thru 0xd1bac3 (0x32c4 bytes) 2024-09-17T23:29:45.720 INFO:teuthology.orchestra.run.smithi139.stdout:1285 punch from 0x751e000 to 0x7522e00, (0x4e00 bytes) 2024-09-17T23:29:45.733 INFO:teuthology.orchestra.run.smithi139.stdout:1286 write 0xe87e800 thru 0xe88645e (0x7c5f bytes) 2024-09-17T23:29:45.733 INFO:teuthology.orchestra.run.smithi139.stdout:1287 punch from 0x1792800 to 0x179e600, (0xbe00 bytes) 2024-09-17T23:29:45.735 INFO:teuthology.orchestra.run.smithi139.stdout:1288 read 0x665200 thru 0x66b9fc (0x67fd bytes) 2024-09-17T23:29:45.736 INFO:teuthology.orchestra.run.smithi139.stdout:1289 read 0x3b1ba00 thru 0x3b1bba2 (0x1a3 bytes) 2024-09-17T23:29:45.737 INFO:teuthology.orchestra.run.smithi139.stdout:1290 read 0xca9a800 thru 0xca9ded1 (0x36d2 bytes) 2024-09-17T23:29:45.737 INFO:teuthology.orchestra.run.smithi139.stdout:1293 read 0x7d4c00 thru 0x7d8f31 (0x4332 bytes) 2024-09-17T23:29:45.737 INFO:teuthology.orchestra.run.smithi139.stdout:1294 write 0xc541a00 thru 0xc54835f (0x6960 bytes) 2024-09-17T23:29:45.737 INFO:teuthology.orchestra.run.smithi139.stdout:1295 write 0x1d19a00 thru 0x1d1af39 (0x153a bytes) 2024-09-17T23:29:45.753 INFO:teuthology.orchestra.run.smithi139.stdout:1296 trunc from 0xe886600 to 0x1fa3000 2024-09-17T23:29:45.772 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:45.769+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:45.783 INFO:teuthology.orchestra.run.smithi139.stdout:1297 read 0x16ab800 thru 0x16b028d (0x4a8e bytes) 2024-09-17T23:29:45.787 INFO:teuthology.orchestra.run.smithi139.stdout:1298 trunc from 0x1fa3000 to 0x5612c00 2024-09-17T23:29:45.795 INFO:teuthology.orchestra.run.smithi139.stdout:1302 write 0x2ca4800 thru 0x2caabd1 (0x63d2 bytes) 2024-09-17T23:29:45.801 INFO:teuthology.orchestra.run.smithi139.stdout:1303 trunc from 0x5612c00 to 0x876be00 2024-09-17T23:29:45.811 INFO:teuthology.orchestra.run.smithi139.stdout:1304 punch from 0x1b12000 to 0x1b18800, (0x6800 bytes) 2024-09-17T23:29:45.817 INFO:teuthology.orchestra.run.smithi139.stdout:1305 read 0x480e800 thru 0x481782e (0x902f bytes) 2024-09-17T23:29:45.825 INFO:teuthology.orchestra.run.smithi139.stdout:1311 trunc from 0x876be00 to 0xd7f9a00 2024-09-17T23:29:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:45.831 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:45.831 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:45.838 INFO:teuthology.orchestra.run.smithi139.stdout:1316 read 0x5fff000 thru 0x6007ca6 (0x8ca7 bytes) 2024-09-17T23:29:45.839 INFO:teuthology.orchestra.run.smithi139.stdout:1317 read 0x2dc1200 thru 0x2dcfa78 (0xe879 bytes) 2024-09-17T23:29:45.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:45.841 INFO:teuthology.orchestra.run.smithi139.stdout:1319 punch from 0x9bfd200 to 0x9c0ce00, (0xfc00 bytes) 2024-09-17T23:29:45.841 INFO:teuthology.orchestra.run.smithi139.stdout:1320 read 0x62cfe00 thru 0x62d4050 (0x4251 bytes) 2024-09-17T23:29:45.841 INFO:teuthology.orchestra.run.smithi139.stdout:1321 read 0x7f4f400 thru 0x7f5b9fb (0xc5fc bytes) 2024-09-17T23:29:45.844 INFO:teuthology.orchestra.run.smithi139.stdout:1323 trunc from 0xd7f9a00 to 0x9c2e400 2024-09-17T23:29:45.861 INFO:teuthology.orchestra.run.smithi139.stdout:1325 punch from 0x85bd400 to 0x85cc600, (0xf200 bytes) 2024-09-17T23:29:45.861 INFO:teuthology.orchestra.run.smithi139.stdout:1329 read 0x883ce00 thru 0x8844a76 (0x7c77 bytes) 2024-09-17T23:29:45.861 INFO:teuthology.orchestra.run.smithi139.stdout:1331 read 0x3174000 thru 0x3174506 (0x507 bytes) 2024-09-17T23:29:45.864 INFO:teuthology.orchestra.run.smithi139.stdout:1333 trunc from 0x9c2e400 to 0x857ca00 2024-09-17T23:29:45.886 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:45.884+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:45.899 INFO:teuthology.orchestra.run.smithi139.stdout:1335 write 0xae08a00 thru 0xae12972 (0x9f73 bytes) 2024-09-17T23:29:45.907 INFO:teuthology.orchestra.run.smithi139.stdout:1336 read 0x6bc8200 thru 0x6bcf5be (0x73bf bytes) 2024-09-17T23:29:45.907 INFO:teuthology.orchestra.run.smithi139.stdout:1337 read 0x49acc00 thru 0x49b1adc (0x4edd bytes) 2024-09-17T23:29:45.925 INFO:teuthology.orchestra.run.smithi139.stdout:1338 write 0xd331800 thru 0xd33e268 (0xca69 bytes) 2024-09-17T23:29:45.932 INFO:teuthology.orchestra.run.smithi139.stdout:1339 write 0x81a6c00 thru 0x81ac376 (0x5777 bytes) 2024-09-17T23:29:45.953 INFO:teuthology.orchestra.run.smithi139.stdout:1340 write 0xd923000 thru 0xd928777 (0x5778 bytes) 2024-09-17T23:29:45.953 INFO:teuthology.orchestra.run.smithi139.stdout:1342 read 0xa3ef600 thru 0xa3fd52e (0xdf2f bytes) 2024-09-17T23:29:45.954 INFO:teuthology.orchestra.run.smithi139.stdout:1343 read 0x859ac00 thru 0x859c95a (0x1d5b bytes) 2024-09-17T23:29:45.954 INFO:teuthology.orchestra.run.smithi139.stdout:1344 write 0x35be00 thru 0x35f8fd (0x3afe bytes) 2024-09-17T23:29:45.954 INFO:teuthology.orchestra.run.smithi139.stdout:1345 read 0xa30fa00 thru 0xa312e88 (0x3489 bytes) 2024-09-17T23:29:45.956 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:45.956 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:45.957 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:45.957 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:45.957 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:45.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:45.968 INFO:teuthology.orchestra.run.smithi139.stdout:1347 trunc from 0xd928800 to 0x1f15200 2024-09-17T23:29:45.974 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:45.971+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:29:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.15 pg_upmap_items mapping to [5->2,1->4] 2024-09-17T23:29:46.019 INFO:teuthology.orchestra.run.smithi139.stdout:1349 write 0x2dc7400 thru 0x2dd6a0f (0xf610 bytes) 2024-09-17T23:29:46.039 INFO:teuthology.orchestra.run.smithi139.stdout:1350 write 0x6e20400 thru 0x6e2e203 (0xde04 bytes) 2024-09-17T23:29:46.039 INFO:teuthology.orchestra.run.smithi139.stdout:1352 read 0x14b9400 thru 0x14bd697 (0x4298 bytes) 2024-09-17T23:29:46.040 INFO:teuthology.orchestra.run.smithi139.stdout:1353 punch from 0x2b97800 to 0x2ba6400, (0xec00 bytes) 2024-09-17T23:29:46.061 INFO:teuthology.orchestra.run.smithi139.stdout:1354 write 0x942ae00 thru 0x94353fd (0xa5fe bytes) 2024-09-17T23:29:46.061 INFO:teuthology.orchestra.run.smithi139.stdout:1355 read 0x3400a00 thru 0x340ffa6 (0xf5a7 bytes) 2024-09-17T23:29:46.062 INFO:teuthology.orchestra.run.smithi139.stdout:1356 read 0x8045600 thru 0x804b5de (0x5fdf bytes) 2024-09-17T23:29:46.062 INFO:teuthology.orchestra.run.smithi139.stdout:1358 write 0xd32c00 thru 0xd3e712 (0xbb13 bytes) 2024-09-17T23:29:46.075 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:46.072+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:46.081 INFO:teuthology.orchestra.run.smithi139.stdout:1360 write 0x98bec00 thru 0x98bfe2b (0x122c bytes) 2024-09-17T23:29:46.082 INFO:teuthology.orchestra.run.smithi139.stdout:1361 read 0x5a65000 thru 0x5a68f2c (0x3f2d bytes) 2024-09-17T23:29:46.097 INFO:teuthology.orchestra.run.smithi139.stdout:1362 write 0x9a8de00 thru 0x9a9c4a4 (0xe6a5 bytes) 2024-09-17T23:29:46.098 INFO:teuthology.orchestra.run.smithi139.stdout:1364 read 0x2ed5200 thru 0x2eded9c (0x9b9d bytes) 2024-09-17T23:29:46.098 INFO:teuthology.orchestra.run.smithi139.stdout:1367 read 0x468200 thru 0x46e4ff (0x6300 bytes) 2024-09-17T23:29:46.103 INFO:teuthology.orchestra.run.smithi139.stdout:1368 read 0x8b37400 thru 0x8b4733e (0xff3f bytes) 2024-09-17T23:29:46.103 INFO:teuthology.orchestra.run.smithi139.stdout:1370 read 0x6b2c200 thru 0x6b3825d (0xc05e bytes) 2024-09-17T23:29:46.111 INFO:teuthology.orchestra.run.smithi139.stdout:1372 trunc from 0x9a9c600 to 0x3613800 2024-09-17T23:29:46.125 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1983 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1805312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6306816479999995, 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048765663000000001, 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1983 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:46.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492092+0000", 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.22e-06 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492095+0000", 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2279999999999998e-06 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492098+0000", 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540732+0000", 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048634183999999997 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540781+0000", 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8976e-05 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540856+0000", 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5142000000000007e-05 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.130 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1984 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1814528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6306710320000004, 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049003441000000002, 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1984 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.131 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:46.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492103+0000", 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1540000000000001e-06 2024-09-17T23:29:46.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492107+0000", 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0249999999999996e-06 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492109+0000", 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540902+0000", 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048793087999999998 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540995+0000", 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2534000000000005e-05 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541105+0000", 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000109815 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1985 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1823744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6306597380000003, 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049142263999999998, 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1985 2024-09-17T23:29:46.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493825+0000", 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017121879999999999 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493829+0000", 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2509999999999996e-06 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493836+0000", 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000002e-06 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493840+0000", 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3220000000000003e-06 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541141+0000", 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047301395000000003 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541190+0000", 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8819999999999997e-05 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541255+0000", 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4587999999999999e-05 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2106 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1096704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6288197800000006, 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797818999999999, 2024-09-17T23:29:46.140 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2106 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1699999999999998e-07 2024-09-17T23:29:46.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493955+0000", 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4059999999999999e-06 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493962+0000", 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6379999999999998e-06 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493968+0000", 2024-09-17T23:29:46.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4149999999999998e-06 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542562+0000", 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048594796000000003 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542650+0000", 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7288999999999997e-05 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542750+0000", 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100758 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2107 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1105920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6287975729999999, 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048932067000000003, 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.145 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2107 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.22e-07 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493977+0000", 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:29:46.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493984+0000", 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0920000000000003e-06 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493989+0000", 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0370000000000002e-06 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542786+0000", 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797034000000003 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542841+0000", 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5781e-05 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542907+0000", 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5362000000000002e-05 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2108 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1115136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6287770429999995, 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049148982000000001, 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2108 2024-09-17T23:29:46.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493996+0000", 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.03e-07 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493998+0000", 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.187e-06 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494004+0000", 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2219999999999999e-06 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494009+0000", 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6870000000000002e-06 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542954+0000", 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048945206999999998 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543043+0000", 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9494999999999996e-05 2024-09-17T23:29:46.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543144+0000", 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100781 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:29:46.154 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6287570529999993, 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:29:46.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:29:46.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:29:46.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.159 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6287371729999993, 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:29:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:29:46.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.164 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6287156350000007, 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.165 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:29:46.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2117 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1198080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6268533200000004, 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048875710000000003, 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2117 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495920+0000", 2024-09-17T23:29:46.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.105e-06 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495924+0000", 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.269e-06 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495926+0000", 2024-09-17T23:29:46.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5830000000000001e-06 2024-09-17T23:29:46.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544618+0000", 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048691940000000003 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544704+0000", 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5581000000000006e-05 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544795+0000", 2024-09-17T23:29:46.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1020999999999995e-05 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2118 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1207296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6268431129999996, 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.04902596, 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.177 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2118 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495931+0000", 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.139e-06 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495934+0000", 2024-09-17T23:29:46.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1870000000000001e-06 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495936+0000", 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.542e-06 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544842+0000", 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048905954000000001 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544893+0000", 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0927999999999998e-05 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544955+0000", 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999000000000001e-05 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.181 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6268330370000008, 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:29:46.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:29:46.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6268229210000005, 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:29:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:29:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:29:46.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.190 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6268106909999993, 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:29:46.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.195 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6268001220000006, 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:29:46.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.200 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6267902339999996, 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:46.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:29:46.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.205 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6267799519999997, 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:29:46.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:29:46.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6267640649999997, 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:29:46.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2128 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1299456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6248792230000006, 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048685818999999998, 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2128 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5800000000000001e-07 2024-09-17T23:29:46.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497895+0000", 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.44e-06 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497905+0000", 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0231000000000001e-05 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497912+0000", 2024-09-17T23:29:46.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8970000000000002e-06 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546386+0000", 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048474585000000001 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546477+0000", 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0301000000000002e-05 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546579+0000", 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102107 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2129 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1308672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.6248561539999997, 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048890621000000002, 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2129 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5100000000000001e-07 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497918+0000", 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4670000000000001e-06 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497922+0000", 2024-09-17T23:29:46.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497924+0000", 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6390000000000001e-06 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546622+0000", 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048697427000000001 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:46.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546706+0000", 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4306000000000002e-05 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546807+0000", 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100851 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:46.223 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:46.225 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:46.222+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:46.225 INFO:teuthology.orchestra.run.smithi139.stdout:1373 write 0x9df1c00 thru 0x9df4fee (0x33ef bytes) 2024-09-17T23:29:46.225 INFO:teuthology.orchestra.run.smithi139.stdout:1374 write 0xbc66000 thru 0xbc725cf (0xc5d0 bytes) 2024-09-17T23:29:46.225 INFO:teuthology.orchestra.run.smithi139.stdout:1377 flatten 2024-09-17T23:29:46.225 INFO:teuthology.orchestra.run.smithi139.stdout:1381 trunc from 0xbc72600 to 0xb49000 2024-09-17T23:29:46.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:46.244 INFO:teuthology.orchestra.run.smithi139.stdout:1382 write 0x619e000 thru 0x61aab90 (0xcb91 bytes) 2024-09-17T23:29:46.245 INFO:teuthology.orchestra.run.smithi139.stdout:1384 read 0x1df2e00 thru 0x1e0127d (0xe47e bytes) 2024-09-17T23:29:46.271 INFO:teuthology.orchestra.run.smithi139.stdout:1386 write 0xc3a3c00 thru 0xc3a43a8 (0x7a9 bytes) 2024-09-17T23:29:46.278 INFO:teuthology.orchestra.run.smithi139.stdout:1388 write 0x34ae600 thru 0x34b651e (0x7f1f bytes) 2024-09-17T23:29:46.281 INFO:teuthology.orchestra.run.smithi139.stdout:1389 trunc from 0xc3a4400 to 0xee14000 2024-09-17T23:29:46.293 INFO:teuthology.orchestra.run.smithi139.stdout:1390 punch from 0x1d64400 to 0x1d67400, (0x3000 bytes) 2024-09-17T23:29:46.302 INFO:teuthology.orchestra.run.smithi139.stdout:1391 read 0xc544a00 thru 0xc54ba8c (0x708d bytes) 2024-09-17T23:29:46.318 INFO:teuthology.orchestra.run.smithi139.stdout:1393 trunc from 0xee14000 to 0x1c43e00 2024-09-17T23:29:46.325 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:46.322+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:46.340 INFO:teuthology.orchestra.run.smithi139.stdout:1394 read 0x1135200 thru 0x114065b (0xb45c bytes) 2024-09-17T23:29:46.340 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:46.340 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:46.340 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:46.340 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:46.344 INFO:teuthology.orchestra.run.smithi139.stdout:1396 clone 13 order 20 su 4096 sc 10 2024-09-17T23:29:46.349 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:46.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:46.426 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:46.424+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:46.456 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:46.457 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:46.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:46.524 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:46.524 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:46.541 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:46.541 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:46.550 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:46.548+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:46.557 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:46.558 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:46.575 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:46.576 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:46.592 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:46.593 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:46.609 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:46.610 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:46.615 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:46.615 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:46.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:46.665 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:46.663+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:46.727 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:46.725+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:46.730 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:46.730 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:46.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:46.828 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:46.826+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:46.885 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:46.885 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:46.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:46.930 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:46.928+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:47.005 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.005 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:47.058 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:47.055+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:47.125 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.125 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.132 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:47.130+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:47.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:47.233 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:47.230+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:47.286 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.287 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:47.334 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:47.332+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:47.394 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T23:29:47.435 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:47.433+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:47.443 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.443 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:47.466 INFO:teuthology.orchestra.run.smithi139.stdout:1397 read 0x1aed800 thru 0x1afb9c1 (0xe1c2 bytes) 2024-09-17T23:29:47.468 INFO:teuthology.orchestra.run.smithi139.stdout:1400 read 0xbca400 thru 0xbd7544 (0xd145 bytes) 2024-09-17T23:29:47.470 INFO:teuthology.orchestra.run.smithi139.stdout:1402 punch from 0x1b1ce00 to 0x1b24000, (0x7200 bytes) 2024-09-17T23:29:47.536 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:47.534+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:47.564 INFO:teuthology.orchestra.run.smithi139.stdout:1404 read 0x9f8000 thru 0x9ff26b (0x726c bytes) 2024-09-17T23:29:47.581 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.582 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.588 INFO:teuthology.orchestra.run.smithi139.stdout:1406 write 0xb3b6800 thru 0xb3b8ac7 (0x22c8 bytes) 2024-09-17T23:29:47.588 INFO:teuthology.orchestra.run.smithi139.stdout:1407 read 0x2aede00 thru 0x2afc5e7 (0xe7e8 bytes) 2024-09-17T23:29:47.588 INFO:teuthology.orchestra.run.smithi139.stdout:1409 punch from 0x1052a00 to 0x1061600, (0xec00 bytes) 2024-09-17T23:29:47.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:47.637 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:47.634+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:47.659 INFO:teuthology.orchestra.run.smithi139.stdout:1411 read 0x23e1000 thru 0x23e3ee1 (0x2ee2 bytes) 2024-09-17T23:29:47.659 INFO:teuthology.orchestra.run.smithi139.stdout:1412 read 0x8155400 thru 0x815a580 (0x5181 bytes) 2024-09-17T23:29:47.662 INFO:teuthology.orchestra.run.smithi139.stdout:1418 trunc from 0xb3b8c00 to 0xe37d200 2024-09-17T23:29:47.737 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:47.736+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.764 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:47.795 INFO:teuthology.orchestra.run.smithi139.stdout:1419 trunc from 0xe37d200 to 0x9906a00 2024-09-17T23:29:47.809 INFO:teuthology.orchestra.run.smithi139.stdout:1421 punch from 0x96ffa00 to 0x9705000, (0x5600 bytes) 2024-09-17T23:29:47.813 INFO:teuthology.orchestra.run.smithi139.stdout:1423 trunc from 0x9906a00 to 0x83a2000 2024-09-17T23:29:47.841 INFO:teuthology.orchestra.run.smithi139.stdout:1425 trunc from 0x83a2000 to 0xc37aa00 2024-09-17T23:29:47.850 INFO:teuthology.orchestra.run.smithi139.stdout:1426 write 0x883c800 thru 0x884a2c5 (0xdac6 bytes) 2024-09-17T23:29:47.850 INFO:teuthology.orchestra.run.smithi139.stdout:1427 read 0xba7e200 thru 0xba82a8b (0x488c bytes) 2024-09-17T23:29:47.857 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:47.855+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:47.873 INFO:teuthology.orchestra.run.smithi139.stdout:1428 write 0xc53e200 thru 0xc54b537 (0xd338 bytes) 2024-09-17T23:29:47.905 INFO:teuthology.orchestra.run.smithi139.stdout:1429 write 0xe874e00 thru 0xe87a2b2 (0x54b3 bytes) 2024-09-17T23:29:47.918 INFO:teuthology.orchestra.run.smithi139.stdout:1431 trunc from 0xe87a400 to 0x302d400 2024-09-17T23:29:47.939 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:47.937+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:47.948 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:47.948 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:47.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:47.972 INFO:teuthology.orchestra.run.smithi139.stdout:1432 trunc from 0x302d400 to 0xe0b1c00 2024-09-17T23:29:47.983 INFO:teuthology.orchestra.run.smithi139.stdout:1434 write 0xa0ec00 thru 0xa1114b (0x254c bytes) 2024-09-17T23:29:47.984 INFO:teuthology.orchestra.run.smithi139.stdout:1437 read 0x8a1fe00 thru 0x8a25339 (0x553a bytes) 2024-09-17T23:29:47.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:29:47.996 INFO:teuthology.orchestra.run.smithi139.stdout:1441 trunc from 0xe0b1c00 to 0x3d80e00 2024-09-17T23:29:48.010 INFO:teuthology.orchestra.run.smithi139.stdout:1444 punch from 0x3216200 to 0x321be00, (0x5c00 bytes) 2024-09-17T23:29:48.013 INFO:teuthology.orchestra.run.smithi139.stdout:1446 clone 14 order 23 su 65536 sc 12 2024-09-17T23:29:48.041 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:48.039+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:48.074 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:48.074 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:48.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:48.167 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:48.165+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:48.230 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:48.230 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:48.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:48.280 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:48.278+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:48.343 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:48.341+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:48.386 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:48.386 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:48.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:48.444 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:48.442+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:48.502 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:48.502 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:48.511 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:48.546 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:48.543+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:48.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:48.657 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:48.658 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:48.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:48.667 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:48.664+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:48.747 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:48.747 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:48.747 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:48.745+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:48.763 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:48.774 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:48.774 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:48.779 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:48.780 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:48.783 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:48.797 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:48.797 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:48.813 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:48.813 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:48.830 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:48.830 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:48.882 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:48.879+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:48.930 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:48.931 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:48.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:48.949 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:48.946+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:48.990 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:29:49.048 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:49.048 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:49.048 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:49.049 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:49.049 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:49.050 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:49.048+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:49.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:49.082 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone13 from 0x3d80e00 (overlap 0x1c43e00) to 0x0 2024-09-17T23:29:49.150 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:49.147+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:49.251 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:49.248+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1795 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234911758999999, 2024-09-17T23:29:49.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017441358000000001, 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1795 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013920+0000", 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.04e-07 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013921+0000", 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9699999999999994e-07 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013924+0000", 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9409999999999999e-06 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013927+0000", 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.435e-06 2024-09-17T23:29:49.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031185+0000", 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017258909999999999 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031263+0000", 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7015999999999996e-05 2024-09-17T23:29:49.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031361+0000", 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8855000000000001e-05 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1796 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15335424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234901325999999, 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017564354000000001, 2024-09-17T23:29:49.257 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1796 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013930+0000", 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013931+0000", 2024-09-17T23:29:49.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-07 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013935+0000", 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5859999999999998e-06 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013939+0000", 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.473e-06 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013941+0000", 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.492e-06 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031391+0000", 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017449416999999998 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031438+0000", 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6885999999999998e-05 2024-09-17T23:29:49.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031495+0000", 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7281999999999998e-05 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1797 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 9830400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234886995, 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017763582999999999, 2024-09-17T23:29:49.262 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1797 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013945+0000", 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2600000000000001e-07 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013946+0000", 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.071e-06 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013949+0000", 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1020000000000001e-06 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013952+0000", 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1039999999999998e-06 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031533+0000", 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017580312000000001 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031612+0000", 2024-09-17T23:29:49.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9287000000000002e-05 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031708+0000", 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6481e-05 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1798 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:49.267 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234876290999999, 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017889677, 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1798 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013955+0000", 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013956+0000", 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3900000000000001e-07 2024-09-17T23:29:49.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013957+0000", 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.093e-06 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013960+0000", 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0989999999999999e-06 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013969+0000", 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.67e-06 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031737+0000", 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017768055000000001 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031784+0000", 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7057e-05 2024-09-17T23:29:49.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031845+0000", 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1464000000000006e-05 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1799 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234847940000002, 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018073763999999999, 2024-09-17T23:29:49.272 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1799 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:49.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013984+0000", 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0900000000000002e-07 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013987+0000", 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.379e-06 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.013994+0000", 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4429999999999997e-06 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014000+0000", 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3469999999999998e-06 2024-09-17T23:29:49.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031883+0000", 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017883546 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.031962+0000", 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9000999999999998e-05 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032058+0000", 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5538999999999994e-05 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.277 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1800 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234822596000001, 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018179992999999998, 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1800 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014009+0000", 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014010+0000", 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8299999999999997e-07 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014012+0000", 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1160000000000002e-06 2024-09-17T23:29:49.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014020+0000", 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3340000000000002e-06 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014025+0000", 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2429999999999996e-06 2024-09-17T23:29:49.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032086+0000", 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018060579 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032132+0000", 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6464000000000001e-05 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032189+0000", 2024-09-17T23:29:49.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6774000000000002e-05 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1801 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 10878976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234798198, 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018361467999999999, 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1801 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014034+0000", 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999999999999999e-07 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014036+0000", 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:29:49.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014044+0000", 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1829999999999994e-06 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014051+0000", 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9260000000000003e-06 2024-09-17T23:29:49.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032228+0000", 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018176906 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032304+0000", 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939000000000006e-05 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032395+0000", 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0845000000000002e-05 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1803 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 11403264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234770931, 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018464566000000002, 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1803 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014061+0000", 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014063+0000", 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0779999999999998e-06 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014078+0000", 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4413000000000001e-05 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014084+0000", 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032429+0000", 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018344972000000001 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032469+0000", 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0179000000000003e-05 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032525+0000", 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6561999999999998e-05 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1805 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 11927552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234728864000001, 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018638358000000001, 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1805 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014103+0000", 2024-09-17T23:29:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4099999999999999e-07 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014106+0000", 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6460000000000002e-06 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014116+0000", 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8689999999999999e-06 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014124+0000", 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2139999999999996e-06 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032564+0000", 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018440070999999999 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032639+0000", 2024-09-17T23:29:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4868000000000002e-05 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032741+0000", 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102149 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1810 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234697541999999, 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018741180999999999, 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1810 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014134+0000", 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014135+0000", 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014144+0000", 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0629999999999995e-06 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014151+0000", 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5059999999999998e-06 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014163+0000", 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1399e-05 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032774+0000", 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018610924000000001 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032818+0000", 2024-09-17T23:29:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3927999999999997e-05 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032875+0000", 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7896000000000003e-05 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1814 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234660140999999, 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018916773000000001, 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1814 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014172+0000", 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6300000000000005e-07 2024-09-17T23:29:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014175+0000", 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4739999999999999e-06 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014182+0000", 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0990000000000001e-06 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014187+0000", 2024-09-17T23:29:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2630000000000003e-06 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032915+0000", 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018727573000000001 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.032990+0000", 2024-09-17T23:29:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5766000000000006e-05 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033088+0000", 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7935000000000003e-05 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1822 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234635735000001, 2024-09-17T23:29:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019019635999999999, 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1822 2024-09-17T23:29:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014196+0000", 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6199999999999998e-07 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014199+0000", 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3309999999999998e-06 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014206+0000", 2024-09-17T23:29:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.148e-06 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014212+0000", 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9490000000000001e-06 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033118+0000", 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018906164 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033159+0000", 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1230999999999999e-05 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033216+0000", 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6351000000000003e-05 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1826 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234605165000001, 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019198232999999999, 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1826 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014227+0000", 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014229+0000", 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8419999999999999e-06 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014233+0000", 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1150000000000004e-06 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014242+0000", 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1900000000000001e-06 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033254+0000", 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019012228999999999 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033328+0000", 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3709999999999997e-05 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033425+0000", 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6754000000000002e-05 2024-09-17T23:29:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1830 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234584948999998, 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019304154, 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1830 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014247+0000", 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.15e-07 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014252+0000", 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0939999999999998e-06 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014257+0000", 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3610000000000003e-06 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014259+0000", 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6350000000000002e-06 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033453+0000", 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019193661000000001 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033495+0000", 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033551+0000", 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168000000000002e-05 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1834 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234568605, 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019503455999999999, 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1834 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014263+0000", 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4499999999999998e-07 2024-09-17T23:29:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014265+0000", 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1400000000000001e-06 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014269+0000", 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6149999999999999e-06 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014272+0000", 2024-09-17T23:29:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5220000000000002e-06 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033591+0000", 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019318887 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033668+0000", 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7348000000000006e-05 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033767+0000", 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8698999999999998e-05 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1837 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 15597568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234555912000001, 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019625881000000001, 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1837 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014276+0000", 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.28e-07 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014277+0000", 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.145e-06 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014280+0000", 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1130000000000001e-06 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014283+0000", 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3369999999999998e-06 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033798+0000", 2024-09-17T23:29:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019515498999999999 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033844+0000", 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5553000000000002e-05 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033902+0000", 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8006e-05 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1839 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234545401999998, 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019823990999999999, 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1839 2024-09-17T23:29:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014286+0000", 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014287+0000", 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014288+0000", 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.133e-06 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014291+0000", 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0479999999999999e-06 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014294+0000", 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9160000000000001e-06 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.033941+0000", 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019646961000000001 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034015+0000", 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4411000000000003e-05 2024-09-17T23:29:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034110+0000", 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5313e-05 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4486.0:1841 3.17 3:eaf43f5e:::rbd_data.11522ab9a303.0000000000000002:4 [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.234534720999999, 2024-09-17T23:29:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019940504000000001, 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4486", 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3933850550", 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1841 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014297+0000", 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2999999999999999e-07 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014298+0000", 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0249999999999999e-06 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014302+0000", 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.014304+0000", 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3209999999999999e-06 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034143+0000", 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019838880999999999 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034183+0000", 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9988999999999999e-05 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:23.034238+0000", 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4833999999999997e-05 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:14772 3.12 3:4a278708:::rbd_data.1152c1142907.000000000000002f:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:40.034843+0000", 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2139885970000002, 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.018280547000000001, 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 14772 2024-09-17T23:29:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034843+0000", 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034843+0000", 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034844+0000", 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3499999999999996e-07 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034847+0000", 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6480000000000001e-06 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034855+0000", 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9670000000000001e-06 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034862+0000", 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9450000000000004e-06 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034911+0000", 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8934999999999998e-05 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.034978+0000", 2024-09-17T23:29:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7001000000000004e-05 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035135+0000", 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000157176 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.038818+0000", 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0036822170000000002 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.053005+0000", 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.014187297 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.053070+0000", 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4719000000000002e-05 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.053124+0000", 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4007e-05 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:14823 3.12 3:48da8422:::rbd_data.1152c1142907.0000000000000049:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:40.035701+0000", 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2131304150000002, 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.019721108000000001, 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 14823 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035701+0000", 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035701+0000", 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035702+0000", 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0399999999999996e-07 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035705+0000", 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6929999999999999e-06 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035711+0000", 2024-09-17T23:29:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7630000000000003e-06 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.035716+0000", 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8219999999999997e-06 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.037718+0000", 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020017839999999999 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.037789+0000", 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1569999999999994e-05 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.037966+0000", 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000176639 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.039484+0000", 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001518078 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.055324+0000", 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.015839539999999999 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.055382+0000", 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8349e-05 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:40.055422+0000", 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0365999999999999e-05 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:49.354 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:49.356 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:49.354+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:49.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:49.456 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:49.453+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:49.466 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:49.466 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:49.467 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:49.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:49.531 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T23:29:49.557 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:49.555+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:49.643 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:49.644 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:49.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:49.696 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:49.693+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:49.701 INFO:teuthology.orchestra.run.smithi139.stdout:1450 read 0xe09c00 thru 0xe10d3e (0x713f bytes) 2024-09-17T23:29:49.708 INFO:teuthology.orchestra.run.smithi139.stdout:1453 trunc from 0x3d80e00 to 0x9086400 2024-09-17T23:29:49.730 INFO:teuthology.orchestra.run.smithi139.stdout:1454 trunc from 0x9086400 to 0x11cbc00 2024-09-17T23:29:49.758 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:49.756+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:49.773 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6827 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1559552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.860869+0000", 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9101652590000002, 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051074896000000002, 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6827 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860869+0000", 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860869+0000", 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860870+0000", 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860872+0000", 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5150000000000001e-06 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860879+0000", 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9280000000000004e-06 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860885+0000", 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.911731+0000", 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050846443999999998 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.911841+0000", 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011001999999999999 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.911944+0000", 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103052 2024-09-17T23:29:49.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6828 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1302528~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.860892+0000", 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9101426740000003, 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051254292, 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6828 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860892+0000", 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860892+0000", 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860892+0000", 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860895+0000", 2024-09-17T23:29:49.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2510000000000001e-06 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860901+0000", 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1979999999999998e-06 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860906+0000", 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8409999999999999e-06 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.911983+0000", 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051077810000000001 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912069+0000", 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5315000000000006e-05 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912146+0000", 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7485999999999996e-05 2024-09-17T23:29:49.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6829 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1568768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.860912+0000", 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9101225580000003, 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051490731999999997, 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6829 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860912+0000", 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860912+0000", 2024-09-17T23:29:49.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860912+0000", 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6399999999999998e-07 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860914+0000", 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860920+0000", 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860925+0000", 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9189999999999998e-06 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912192+0000", 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051267213999999998 2024-09-17T23:29:49.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912303+0000", 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110053 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912403+0000", 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010023599999999999 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6830 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1311744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.860932+0000", 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.910102781, 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051676571999999997, 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6830 2024-09-17T23:29:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860932+0000", 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860932+0000", 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860932+0000", 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860934+0000", 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1160000000000002e-06 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860940+0000", 2024-09-17T23:29:49.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860946+0000", 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2889999999999997e-06 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912442+0000", 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051496161999999998 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912525+0000", 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2940000000000002e-05 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912608+0000", 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3629999999999997e-05 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6831 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1577984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.860966+0000", 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9100683289999996, 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051851796999999998, 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6831 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860966+0000", 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860966+0000", 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860967+0000", 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8500000000000001e-07 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860971+0000", 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2799999999999997e-06 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860982+0000", 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0458999999999999e-05 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.860991+0000", 2024-09-17T23:29:49.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3929999999999997e-06 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912646+0000", 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051654971000000001 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912733+0000", 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7398999999999994e-05 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912818+0000", 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4610000000000005e-05 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6832 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1320960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.861003+0000", 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9100313230000001, 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052015000999999998, 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6832 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.861003+0000", 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.861003+0000", 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862582+0000", 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015791360000000001 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862586+0000", 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6720000000000002e-06 2024-09-17T23:29:49.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862598+0000", 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1472e-05 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862605+0000", 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0360000000000001e-06 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912855+0000", 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050250755000000001 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.912942+0000", 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6798999999999993e-05 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913018+0000", 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6130999999999993e-05 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6836 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1596416~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.862703+0000", 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9083312389999998, 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050956741, 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6836 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862703+0000", 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862703+0000", 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862704+0000", 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862707+0000", 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8669999999999998e-06 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862715+0000", 2024-09-17T23:29:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4950000000000008e-06 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862720+0000", 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3859999999999998e-06 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913492+0000", 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050771595000000003 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913586+0000", 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3859999999999999e-05 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913660+0000", 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4176000000000003e-05 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6837 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1348608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.862728+0000", 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.908307035, 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051177677999999997, 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6837 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862728+0000", 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862728+0000", 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862728+0000", 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862730+0000", 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2230000000000001e-06 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862736+0000", 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-09-17T23:29:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862754+0000", 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7649999999999999e-05 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913707+0000", 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050953462999999997 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913812+0000", 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104597 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913905+0000", 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3270000000000007e-05 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6838 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.862764+0000", 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9082701359999996, 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051343145, 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6838 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862764+0000", 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862764+0000", 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862765+0000", 2024-09-17T23:29:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5400000000000002e-07 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862773+0000", 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2050000000000002e-06 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862780+0000", 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862785+0000", 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3129999999999998e-06 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.913945+0000", 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051159899000000002 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914029+0000", 2024-09-17T23:29:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4452999999999999e-05 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914108+0000", 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8114999999999993e-05 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6839 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1357824~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.862793+0000", 2024-09-17T23:29:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.908241286, 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051575681999999998, 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6839 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862793+0000", 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862793+0000", 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862794+0000", 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-17T23:29:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862796+0000", 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2259999999999999e-06 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862806+0000", 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8530000000000004e-06 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862810+0000", 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5619999999999997e-06 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914155+0000", 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051344242999999998 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914267+0000", 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011244 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914369+0000", 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101938 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6840 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1614848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.862815+0000", 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9082193780000001, 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051761598999999998, 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6840 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862815+0000", 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862815+0000", 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862815+0000", 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8000000000000002e-07 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862817+0000", 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5269999999999999e-06 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862820+0000", 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.365e-06 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862823+0000", 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6180000000000002e-06 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914407+0000", 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051583570000000002 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914490+0000", 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3929000000000002e-05 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914577+0000", 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6310000000000005e-05 2024-09-17T23:29:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6841 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1367040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.862826+0000", 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9082084760000004, 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051940994999999997, 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6841 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862826+0000", 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.862826+0000", 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864566+0000", 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001739745 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864570+0000", 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7759999999999999e-06 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864582+0000", 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2448e-05 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864589+0000", 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3339999999999996e-06 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914607+0000", 2024-09-17T23:29:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050017963999999998 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914691+0000", 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3700000000000002e-05 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.914767+0000", 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6027999999999998e-05 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6847 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1394688~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.864746+0000", 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9062882099999996, 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050998608000000001, 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6847 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864746+0000", 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864746+0000", 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864747+0000", 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864749+0000", 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4849999999999999e-06 2024-09-17T23:29:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864761+0000", 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1832e-05 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864768+0000", 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1489999999999997e-06 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915581+0000", 2024-09-17T23:29:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050812325999999998 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915667+0000", 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6087000000000004e-05 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915745+0000", 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8274000000000003e-05 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6848 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1651712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.864782+0000", 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9062524789999999, 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051081665999999998, 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6848 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864782+0000", 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864782+0000", 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864783+0000", 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1200000000000003e-07 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864785+0000", 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7580000000000001e-06 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864794+0000", 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2190000000000004e-06 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864799+0000", 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5589999999999996e-06 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915775+0000", 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050975722000000001 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915818+0000", 2024-09-17T23:29:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3545999999999999e-05 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915864+0000", 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5349999999999998e-05 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6849 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1403904~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.864812+0000", 2024-09-17T23:29:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9062229610000001, 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051252930000000002, 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6849 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864812+0000", 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864812+0000", 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864812+0000", 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864814+0000", 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6509999999999999e-06 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864818+0000", 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0949999999999998e-06 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864820+0000", 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7580000000000001e-06 2024-09-17T23:29:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915903+0000", 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051082238000000002 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.915988+0000", 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5240000000000004e-05 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916065+0000", 2024-09-17T23:29:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6648000000000002e-05 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6850 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1660928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.864824+0000", 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9062101689999995, 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051359015000000001, 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6850 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864824+0000", 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864824+0000", 2024-09-17T23:29:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864825+0000", 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999998e-07 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864830+0000", 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3129999999999998e-06 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864833+0000", 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.399e-06 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864836+0000", 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916095+0000", 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051258690000000003 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916139+0000", 2024-09-17T23:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4150999999999998e-05 2024-09-17T23:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916183+0000", 2024-09-17T23:29:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4623999999999998e-05 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6851 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1413120~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.864840+0000", 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9061947889999997, 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051540985999999997, 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6851 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864840+0000", 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.864840+0000", 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866566+0000", 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017260590000000001 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866570+0000", 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1049999999999997e-06 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866579+0000", 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9900000000000003e-06 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866587+0000", 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8730000000000007e-06 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916221+0000", 2024-09-17T23:29:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049633837 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916304+0000", 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3374999999999994e-05 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.916381+0000", 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6747000000000002e-05 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6859 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1449984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.866762+0000", 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9042729889999999, 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050914327000000002, 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6859 2024-09-17T23:29:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866762+0000", 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866762+0000", 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866762+0000", 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866764+0000", 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1280000000000002e-06 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866771+0000", 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1670000000000002e-06 2024-09-17T23:29:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866776+0000", 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9259999999999999e-06 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917507+0000", 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050730771000000001 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917599+0000", 2024-09-17T23:29:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2153999999999999e-05 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917676+0000", 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6777e-05 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6860 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1707008~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.866792+0000", 2024-09-17T23:29:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9042422659999998, 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051001779999999997, 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6860 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866792+0000", 2024-09-17T23:29:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866792+0000", 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866793+0000", 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:29:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866794+0000", 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6509999999999999e-06 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866798+0000", 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2139999999999998e-06 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866801+0000", 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8100000000000002e-06 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917705+0000", 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050904049 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917749+0000", 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4070000000000003e-05 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917794+0000", 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4691000000000003e-05 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4773.0:6861 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 1459200~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:41.866805+0000", 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9042294289999999, 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051184340000000002, 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4773", 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6861 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866805+0000", 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.866805+0000", 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.868566+0000", 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017608980000000001 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.868570+0000", 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1280000000000001e-06 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.868579+0000", 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8489999999999995e-06 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.868588+0000", 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917831+0000", 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049243000000000002 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917912+0000", 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1168999999999995e-05 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:41.917989+0000", 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7069000000000004e-05 2024-09-17T23:29:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:49.872 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:49.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:49.874 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:49.871+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:49.966 INFO:teuthology.orchestra.run.smithi139.stdout:1456 trunc from 0x11cbc00 to 0x1a81200 2024-09-17T23:29:49.974 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:49.972+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:49.984 INFO:teuthology.orchestra.run.smithi139.stdout:1459 trunc from 0x1a81200 to 0x5e7d600 2024-09-17T23:29:50.004 INFO:teuthology.orchestra.run.smithi139.stdout:1460 write 0x6637800 thru 0x663e121 (0x6922 bytes) 2024-09-17T23:29:50.005 INFO:teuthology.orchestra.run.smithi139.stdout:1462 write 0x3da5800 thru 0x3dabeb4 (0x66b5 bytes) 2024-09-17T23:29:50.005 INFO:teuthology.orchestra.run.smithi139.stdout:1464 punch from 0x5b37e00 to 0x5b3e400, (0x6600 bytes) 2024-09-17T23:29:50.008 INFO:teuthology.orchestra.run.smithi139.stdout:1465 trunc from 0x663e200 to 0x82fe000 2024-09-17T23:29:50.017 INFO:teuthology.orchestra.run.smithi139.stdout:1466 read 0x1ed9c00 thru 0x1edcb82 (0x2f83 bytes) 2024-09-17T23:29:50.019 INFO:teuthology.orchestra.run.smithi139.stdout:1467 read 0x19ae400 thru 0x19ae7be (0x3bf bytes) 2024-09-17T23:29:50.020 INFO:teuthology.orchestra.run.smithi139.stdout:1468 write 0xd56000 thru 0xd5e46a (0x846b bytes) 2024-09-17T23:29:50.022 INFO:teuthology.orchestra.run.smithi139.stdout:1469 punch from 0x49e0400 to 0x49e9e00, (0x9a00 bytes) 2024-09-17T23:29:50.022 INFO:teuthology.orchestra.run.smithi139.stdout:1470 write 0x2313200 thru 0x2315a00 (0x2801 bytes) 2024-09-17T23:29:50.025 INFO:teuthology.orchestra.run.smithi139.stdout:1471 punch from 0x3280600 to 0x328e200, (0xdc00 bytes) 2024-09-17T23:29:50.025 INFO:teuthology.orchestra.run.smithi139.stdout:1472 read 0x6d05800 thru 0x6d0648e (0xc8f bytes) 2024-09-17T23:29:50.025 INFO:teuthology.orchestra.run.smithi139.stdout:1473 write 0x2ff3800 thru 0x2ff449e (0xc9f bytes) 2024-09-17T23:29:50.025 INFO:teuthology.orchestra.run.smithi139.stdout:1475 punch from 0x5dbb000 to 0x5dbc800, (0x1800 bytes) 2024-09-17T23:29:50.031 INFO:teuthology.orchestra.run.smithi139.stdout:1477 trunc from 0x82fe000 to 0x391c800 2024-09-17T23:29:50.040 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:50.041 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:50.041 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:50.046 INFO:teuthology.orchestra.run.smithi139.stdout:1478 write 0x249fa00 thru 0x24a940e (0x9a0f bytes) 2024-09-17T23:29:50.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:50.075 INFO:teuthology.orchestra.run.smithi139.stdout:1480 write 0xdd0aa00 thru 0xdd19058 (0xe659 bytes) 2024-09-17T23:29:50.076 INFO:teuthology.orchestra.run.smithi139.stdout:1481 write 0xc882e00 thru 0xc88477e (0x197f bytes) 2024-09-17T23:29:50.090 INFO:teuthology.orchestra.run.smithi139.stdout:1482 trunc from 0xdd19200 to 0x2400c00 2024-09-17T23:29:50.093 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:50.090+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:50.129 INFO:teuthology.orchestra.run.smithi139.stdout:1483 write 0x4cece00 thru 0x4cf142f (0x4630 bytes) 2024-09-17T23:29:50.137 INFO:teuthology.orchestra.run.smithi139.stdout:1484 write 0x1d63400 thru 0x1d638d7 (0x4d8 bytes) 2024-09-17T23:29:50.137 INFO:teuthology.orchestra.run.smithi139.stdout:1487 punch from 0xb50c00 to 0xb58800, (0x7c00 bytes) 2024-09-17T23:29:50.138 INFO:teuthology.orchestra.run.smithi139.stdout:1488 read 0x4457000 thru 0x446454a (0xd54b bytes) 2024-09-17T23:29:50.149 INFO:teuthology.orchestra.run.smithi139.stdout:1489 write 0x5d7be00 thru 0x5d806a1 (0x48a2 bytes) 2024-09-17T23:29:50.168 INFO:teuthology.orchestra.run.smithi139.stdout:1491 write 0xc101600 thru 0xc1078d5 (0x62d6 bytes) 2024-09-17T23:29:50.173 INFO:teuthology.orchestra.run.smithi139.stdout:1492 punch from 0x6208800 to 0x6217600, (0xee00 bytes) 2024-09-17T23:29:50.176 INFO:teuthology.orchestra.run.smithi139.stdout:1495 punch from 0x6362600 to 0x6365200, (0x2c00 bytes) 2024-09-17T23:29:50.176 INFO:teuthology.orchestra.run.smithi139.stdout:1498 punch from 0x30b5c00 to 0x30bb000, (0x5400 bytes) 2024-09-17T23:29:50.176 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:50.174+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:50.183 INFO:teuthology.orchestra.run.smithi139.stdout:1501 trunc from 0xc107a00 to 0x587be00 2024-09-17T23:29:50.208 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:50.209 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:50.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:50.222 INFO:teuthology.orchestra.run.smithi139.stdout:1505 write 0xcf97c00 thru 0xcfa6690 (0xea91 bytes) 2024-09-17T23:29:50.223 INFO:teuthology.orchestra.run.smithi139.stdout:1506 read 0x418dc00 thru 0x419256e (0x496f bytes) 2024-09-17T23:29:50.229 INFO:teuthology.orchestra.run.smithi139.stdout:1507 trunc from 0xcfa6800 to 0xa943800 2024-09-17T23:29:50.254 INFO:teuthology.orchestra.run.smithi139.stdout:1510 trunc from 0xa943800 to 0x41dcc00 2024-09-17T23:29:50.266 INFO:teuthology.orchestra.run.smithi139.stdout:1511 read 0x1636a00 thru 0x1642be8 (0xc1e9 bytes) 2024-09-17T23:29:50.267 INFO:teuthology.orchestra.run.smithi139.stdout:1513 punch from 0x221c600 to 0x2225600, (0x9000 bytes) 2024-09-17T23:29:50.268 INFO:teuthology.orchestra.run.smithi139.stdout:1514 punch from 0x496400 to 0x49fc00, (0x9800 bytes) 2024-09-17T23:29:50.269 INFO:teuthology.orchestra.run.smithi139.stdout:1518 read 0x3be6e00 thru 0x3bf24dc (0xb6dd bytes) 2024-09-17T23:29:50.274 INFO:teuthology.orchestra.run.smithi139.stdout:1519 trunc from 0x41dcc00 to 0x8534600 2024-09-17T23:29:50.277 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:50.275+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:50.285 INFO:teuthology.orchestra.run.smithi139.stdout:1520 read 0x28aba00 thru 0x28b371e (0x7d1f bytes) 2024-09-17T23:29:50.300 INFO:teuthology.orchestra.run.smithi139.stdout:1521 write 0xbf97a00 thru 0xbf982d8 (0x8d9 bytes) 2024-09-17T23:29:50.301 INFO:teuthology.orchestra.run.smithi139.stdout:1524 read 0x3a84c00 thru 0x3a85a34 (0xe35 bytes) 2024-09-17T23:29:50.305 INFO:teuthology.orchestra.run.smithi139.stdout:1527 trunc from 0xbf98400 to 0xa8ac400 2024-09-17T23:29:50.320 INFO:teuthology.orchestra.run.smithi139.stdout:1528 punch from 0x8188800 to 0x8194400, (0xbc00 bytes) 2024-09-17T23:29:50.338 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:50.338 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:50.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.863868283000002, 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:50.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:50.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:50.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1061 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 657408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.890801114, 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037158380999999997, 2024-09-17T23:29:50.346 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1061 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444507+0000", 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000001e-07 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444509+0000", 2024-09-17T23:29:50.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7129999999999999e-06 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444512+0000", 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2059999999999999e-06 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444515+0000", 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481580+0000", 2024-09-17T23:29:50.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037065396 2024-09-17T23:29:50.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481625+0000", 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5049000000000002e-05 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481665+0000", 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9949e-05 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1062 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 666624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.890790018000001, 2024-09-17T23:29:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037282721999999997, 2024-09-17T23:29:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1062 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444518+0000", 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0100000000000001e-07 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444519+0000", 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0219999999999999e-06 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444522+0000", 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1360000000000001e-06 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444526+0000", 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6950000000000002e-06 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481709+0000", 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037183131000000001 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481761+0000", 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1768000000000003e-05 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481801+0000", 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9768999999999998e-05 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1063 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 675840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.890778947999999, 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037384175999999998, 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1063 2024-09-17T23:29:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444529+0000", 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444784+0000", 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025496399999999999 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444788+0000", 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0439999999999997e-06 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444794+0000", 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5019999999999998e-06 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444798+0000", 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8959999999999996e-06 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481829+0000", 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037030346999999998 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481875+0000", 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6490999999999999e-05 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481913+0000", 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7932000000000002e-05 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1064 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 685056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.890497229999999, 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037215831999999997, 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1064 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444811+0000", 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7700000000000001e-07 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444821+0000", 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.046e-05 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444826+0000", 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7380000000000002e-06 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444829+0000", 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2169999999999999e-06 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481946+0000", 2024-09-17T23:29:50.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037116984999999998 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.481989+0000", 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2589999999999997e-05 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482026+0000", 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7564999999999997e-05 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1065 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 694272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.890457711, 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037301835999999998, 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1065 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444850+0000", 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444851+0000", 2024-09-17T23:29:50.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444852+0000", 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8819999999999999e-06 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444856+0000", 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0849999999999999e-06 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444859+0000", 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9919999999999999e-06 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482057+0000", 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037197539000000002 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482107+0000", 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9530000000000002e-05 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482152+0000", 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5419e-05 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1066 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 703488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.890443663999999, 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037415125, 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1066 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.444864+0000", 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445344+0000", 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047944399999999999 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445346+0000", 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-09-17T23:29:50.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445359+0000", 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2573e-05 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445363+0000", 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482181+0000", 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036818371000000003 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482225+0000", 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3591000000000003e-05 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482279+0000", 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4447000000000002e-05 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1069 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 731136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.889896482999999, 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037253597999999999, 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1069 2024-09-17T23:29:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445411+0000", 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445412+0000", 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.72e-07 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445413+0000", 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2780000000000001e-06 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445416+0000", 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3529999999999999e-06 2024-09-17T23:29:50.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445425+0000", 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0119999999999994e-06 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482567+0000", 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037141277 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482618+0000", 2024-09-17T23:29:50.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1445e-05 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482665+0000", 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6961e-05 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1070 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 740352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.889865391000001, 2024-09-17T23:29:50.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037344291000000002, 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1070 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445442+0000", 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445443+0000", 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445446+0000", 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2480000000000001e-06 2024-09-17T23:29:50.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445452+0000", 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5659999999999997e-06 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445455+0000", 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3560000000000001e-06 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482693+0000", 2024-09-17T23:29:50.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037238009000000002 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482745+0000", 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2382000000000001e-05 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482787+0000", 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1337000000000001e-05 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1071 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 749568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.889847700000001, 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037450310000000001, 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1071 2024-09-17T23:29:50.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445460+0000", 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445528+0000", 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7896000000000002e-05 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445530+0000", 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2630000000000002e-06 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445536+0000", 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9660000000000001e-06 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445541+0000", 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7709999999999997e-06 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482816+0000", 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037275413 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482864+0000", 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7750999999999997e-05 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482910+0000", 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6249999999999999e-05 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1072 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 758784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.889751690000001, 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037482326000000003, 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1072 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445556+0000", 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445635+0000", 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9115000000000004e-05 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445638+0000", 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.779e-06 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445643+0000", 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4410000000000001e-06 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445647+0000", 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.794e-06 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482937+0000", 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037289309 2024-09-17T23:29:50.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.482988+0000", 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1374e-05 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483038+0000", 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0513999999999998e-05 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1073 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 768000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.889652722999999, 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037507367, 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1073 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445655+0000", 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445741+0000", 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5687999999999997e-05 2024-09-17T23:29:50.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445743+0000", 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6199999999999999e-06 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445749+0000", 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4319999999999998e-06 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445752+0000", 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.568e-06 2024-09-17T23:29:50.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483069+0000", 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037316301000000003 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483114+0000", 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999000000000001e-05 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483162+0000", 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8758999999999999e-05 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1074 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.889549257000001, 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037534672999999998, 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1074 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445759+0000", 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445855+0000", 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6780999999999994e-05 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445858+0000", 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6599999999999999e-06 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445864+0000", 2024-09-17T23:29:50.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2500000000000003e-06 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445869+0000", 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7400000000000004e-06 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483192+0000", 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037322704999999998 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483244+0000", 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2744999999999997e-05 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483293+0000", 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8792000000000003e-05 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1075 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:50.401 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.88943218, 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037543656000000002, 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1075 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:50.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.445876+0000", 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446843+0000", 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096762300000000003 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446852+0000", 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9439999999999994e-06 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446861+0000", 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.541e-06 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.446871+0000", 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0498e-05 2024-09-17T23:29:50.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483324+0000", 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036452248999999999 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483375+0000", 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1780000000000002e-05 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.483419+0000", 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020999999999997e-05 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1083 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 860160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.888164449, 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037242256000000001, 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1083 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447143+0000", 2024-09-17T23:29:50.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447144+0000", 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.16e-07 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447151+0000", 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9450000000000004e-06 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447154+0000", 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8639999999999998e-06 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447159+0000", 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484301+0000", 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037141835999999998 2024-09-17T23:29:50.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484347+0000", 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5998000000000002e-05 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484386+0000", 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9060000000000002e-05 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1084 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 869376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.888143725000001, 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037333979000000003, 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1084 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447164+0000", 2024-09-17T23:29:50.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.67e-07 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447166+0000", 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2500000000000001e-06 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447169+0000", 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3189999999999999e-06 2024-09-17T23:29:50.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447172+0000", 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484414+0000", 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037242311 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484458+0000", 2024-09-17T23:29:50.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4515000000000003e-05 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484498+0000", 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9703000000000003e-05 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1085 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 878592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.888122578000001, 2024-09-17T23:29:50.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037436900000000002, 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1085 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:50.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447185+0000", 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447300+0000", 2024-09-17T23:29:50.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011436 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447304+0000", 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2780000000000004e-06 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447312+0000", 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3440000000000001e-06 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447329+0000", 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6307000000000002e-05 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484526+0000", 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037196976999999999 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484581+0000", 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5019999999999998e-05 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484622+0000", 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1613999999999998e-05 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1086 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 887808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.887960948, 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03739141, 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1086 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447347+0000", 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447532+0000", 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018467099999999999 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447536+0000", 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0899999999999998e-06 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447544+0000", 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2859999999999999e-06 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447565+0000", 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0706000000000001e-05 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484652+0000", 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037086898 2024-09-17T23:29:50.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484696+0000", 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4159000000000002e-05 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484738+0000", 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-05 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1087 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 897024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.887729855, 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037273000000000001, 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1087 2024-09-17T23:29:50.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447578+0000", 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447711+0000", 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013340800000000001 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447715+0000", 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9929999999999997e-06 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447724+0000", 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6009999999999995e-06 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447730+0000", 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.852e-06 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484768+0000", 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037037887999999998 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484811+0000", 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3736999999999998e-05 2024-09-17T23:29:50.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484851+0000", 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9521000000000003e-05 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:1088 3.1c 3:3a4c3e0c:::rbd_data.115258a86553.0000000000000005:b [read 906240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.887557791000001, 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037212453999999999, 2024-09-17T23:29:50.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1088 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447750+0000", 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447893+0000", 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000142993 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447901+0000", 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8930000000000005e-06 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447907+0000", 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3300000000000004e-06 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.447911+0000", 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1910000000000002e-06 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484880+0000", 2024-09-17T23:29:50.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.036968207000000003 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484923+0000", 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3197e-05 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:37.484962+0000", 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9643e-05 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.434 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:50.435 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:50.433+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:50.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:50.437 INFO:teuthology.orchestra.run.smithi139.stdout:1529 write 0xc1bd200 thru 0xc1c931f (0xc120 bytes) 2024-09-17T23:29:50.437 INFO:teuthology.orchestra.run.smithi139.stdout:1530 write 0x55c7c00 thru 0x55c80d9 (0x4da bytes) 2024-09-17T23:29:50.437 INFO:teuthology.orchestra.run.smithi139.stdout:1532 trunc from 0xc1c9400 to 0xd66f600 2024-09-17T23:29:50.437 INFO:teuthology.orchestra.run.smithi139.stdout:1534 write 0x2856a00 thru 0x2859823 (0x2e24 bytes) 2024-09-17T23:29:50.437 INFO:teuthology.orchestra.run.smithi139.stdout:1535 read 0x8457c00 thru 0x8462fe9 (0xb3ea bytes) 2024-09-17T23:29:50.437 INFO:teuthology.orchestra.run.smithi139.stdout:1543 write 0x902c200 thru 0x9032404 (0x6205 bytes) 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1545 write 0x2462000 thru 0x2467523 (0x5524 bytes) 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1546 read 0x9aeba00 thru 0x9af2b4d (0x714e bytes) 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1547 trunc from 0xd66f600 to 0x54b1000 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1551 write 0x43ca00 thru 0x44c3cb (0xf9cc bytes) 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1555 trunc from 0x54b1000 to 0xf59c00 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1558 read 0x7b3600 thru 0x7c1908 (0xe309 bytes) 2024-09-17T23:29:50.438 INFO:teuthology.orchestra.run.smithi139.stdout:1560 write 0x3eda600 thru 0x3edd4bd (0x2ebe bytes) 2024-09-17T23:29:50.443 INFO:teuthology.orchestra.run.smithi139.stdout:1562 punch from 0x8ad800 to 0x8ba600, (0xce00 bytes) 2024-09-17T23:29:50.450 INFO:teuthology.orchestra.run.smithi139.stdout:1564 trunc from 0x3edd600 to 0x9836600 2024-09-17T23:29:50.466 INFO:teuthology.orchestra.run.smithi139.stdout:1571 trunc from 0x9836600 to 0xe2bb000 2024-09-17T23:29:50.475 INFO:teuthology.orchestra.run.smithi139.stdout:1572 read 0x5fa4600 thru 0x5fafe9e (0xb89f bytes) 2024-09-17T23:29:50.481 INFO:teuthology.orchestra.run.smithi139.stdout:1574 clone 15 order 19 su 8192 sc 9 2024-09-17T23:29:50.536 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:50.534+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:50.584 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:50.585 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:50.585 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:50.585 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:50.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:50.637 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:50.635+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:50.700 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:50.701 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:50.701 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:50.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:50.738 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:50.736+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:50.839 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:50.837+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:50.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14717 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:50.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.81970484, 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007005636, 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14717 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:50.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034530+0000", 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.624e-06 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034535+0000", 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0170000000000004e-06 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034553+0000", 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034567+0000", 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4446e-05 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034630+0000", 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2283e-05 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034770+0000", 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014049000000000001 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041534+0000", 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067642350000000004 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14740 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.819646128, 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070617650000000002, 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14740 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9999999999999997e-07 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034592+0000", 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4010000000000001e-06 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034602+0000", 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0638e-05 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034608+0000", 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.241e-06 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035105+0000", 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049700000000000005 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035244+0000", 2024-09-17T23:29:50.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013833000000000001 2024-09-17T23:29:50.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041649+0000", 2024-09-17T23:29:50.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064055550000000003 2024-09-17T23:29:50.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14760 3.11 3:8ddf073a:::rbd_data.1152c1142907.0000000000000029:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:50.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.819611717000001, 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010811016999999999, 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14760 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034623+0000", 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:29:50.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034627+0000", 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4680000000000003e-06 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034635+0000", 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034641+0000", 2024-09-17T23:29:50.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035590+0000", 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094941100000000005 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036421+0000", 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083071299999999996 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036612+0000", 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191342 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041698+0000", 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050861980000000001 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045354+0000", 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036561440000000001 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045394+0000", 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9604999999999998e-05 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045433+0000", 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8945000000000001e-05 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.818577782, 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:29:50.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:29:50.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:29:50.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.818404842, 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:50.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:29:50.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:29:50.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.818364971999999, 2024-09-17T23:29:50.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:50.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14758 3.1 e41/33 3:853c0e15:::rbd_data.1152c1142907.0000000000000028:head v 41'194, mlcod=39'189)", 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.817000602, 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010716488999999999, 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037234+0000", 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5000000000000002e-07 2024-09-17T23:29:50.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037239+0000", 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.784e-06 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037249+0000", 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8889999999999997e-06 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037257+0000", 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.379e-06 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038255+0000", 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099873299999999991 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038288+0000", 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292000000000002e-05 2024-09-17T23:29:50.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047928+0000", 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096399710000000007 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047950+0000", 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1790999999999999e-05 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14752 3.5 e41/36 3:a0bd3f31:::rbd_data.1152c1142907.0000000000000025:head v 41'34, mlcod=39'31)", 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.815146604000001, 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081097770000000003, 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:50.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039096+0000", 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2230000000000007e-06 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039103+0000", 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7370000000000006e-06 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039109+0000", 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-17T23:29:50.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039160+0000", 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1072999999999997e-05 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039178+0000", 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8771e-05 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047154+0000", 2024-09-17T23:29:50.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079759010000000005 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047197+0000", 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2457999999999999e-05 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14838 3.3 e41/30 3:c39663f8:::rbd_data.1152c1142907.0000000000000064:head v 41'27, mlcod=39'22)", 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:50.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.811493142, 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071267700000000002, 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5300000000000003e-07 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042747+0000", 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-17T23:29:50.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042759+0000", 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2138e-05 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042766+0000", 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979999999999997e-06 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042818+0000", 2024-09-17T23:29:50.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2191000000000002e-05 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042840+0000", 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1535e-05 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049823+0000", 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069831559999999999 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049867+0000", 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.439e-05 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14800 3.1 e41/33 3:8381b477:::rbd_data.1152c1142907.000000000000003d:head v 41'198, mlcod=39'189)", 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.809292785, 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007659412, 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044942+0000", 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-07 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044947+0000", 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.789e-06 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044958+0000", 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0223e-05 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044966+0000", 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999994e-06 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045312+0000", 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034605399999999999 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045344+0000", 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1999999999999999e-05 2024-09-17T23:29:50.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052554+0000", 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072101120000000003 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052600+0000", 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6521999999999999e-05 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21686 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.527053969000001, 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010211289, 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21686 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327182+0000", 2024-09-17T23:29:50.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.199e-06 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327654+0000", 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000472094 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327668+0000", 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3867e-05 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327676+0000", 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9289999999999998e-06 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327897+0000", 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022101 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327979+0000", 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1921000000000004e-05 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328181+0000", 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020200799999999999 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337270+0000", 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090895820000000006 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337286+0000", 2024-09-17T23:29:50.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6112000000000001e-05 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337344+0000", 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7989999999999999e-05 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337391+0000", 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6576999999999998e-05 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21687 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.526532828000001, 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098426639999999992, 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21687 2024-09-17T23:29:50.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5700000000000002e-07 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327704+0000", 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9349999999999999e-06 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327713+0000", 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-06 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327719+0000", 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9059999999999999e-06 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328700+0000", 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00098110899999999997 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328774+0000", 2024-09-17T23:29:50.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3796999999999998e-05 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328937+0000", 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162934 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337441+0000", 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085044609999999996 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337457+0000", 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6378000000000001e-05 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337503+0000", 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5546000000000001e-05 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337543+0000", 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0559999999999998e-05 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21676 3.1 e41/33 3:87de3167:::rbd_data.1152c1142907.0000000000000151:head v 41'200, mlcod=41'196)", 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.526493188, 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097022959999999991, 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327741+0000", 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:29:50.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327746+0000", 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327754+0000", 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999992e-06 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327760+0000", 2024-09-17T23:29:50.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327819+0000", 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9691000000000002e-05 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327838+0000", 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8691000000000001e-05 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337396+0000", 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095580470000000001 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337443+0000", 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6629999999999999e-05 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21694 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.525543651, 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090673060000000007, 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21694 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328691+0000", 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328977+0000", 2024-09-17T23:29:50.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028580700000000001 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328995+0000", 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329004+0000", 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3410000000000003e-06 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330527+0000", 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001522851 2024-09-17T23:29:50.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330615+0000", 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8366000000000001e-05 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330857+0000", 2024-09-17T23:29:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024195999999999999 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337600+0000", 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067435300000000002 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337662+0000", 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1246000000000003e-05 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337721+0000", 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8860000000000002e-05 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337757+0000", 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6749999999999999e-05 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21695 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.525041485999999, 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010154599, 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21695 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329193+0000", 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4499999999999995e-07 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329198+0000", 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329213+0000", 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4606999999999999e-05 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329220+0000", 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.328e-06 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331593+0000", 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002373287 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331670+0000", 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6156999999999996e-05 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331868+0000", 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019856600000000001 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337649+0000", 2024-09-17T23:29:50.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057810889999999997 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339251+0000", 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016017309999999999 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339305+0000", 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3594000000000002e-05 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339347+0000", 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999997e-05 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21693 3.8 e41/30 3:12e31d2e:::rbd_data.1152c1142907.000000000000001a:head v 41'153, mlcod=41'146)", 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:50.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.524625706, 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072668539999999997, 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329609+0000", 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8899999999999999e-07 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329614+0000", 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5639999999999996e-06 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329623+0000", 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329633+0000", 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0003000000000001e-05 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331519+0000", 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018852509999999999 2024-09-17T23:29:50.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331536+0000", 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7393000000000001e-05 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336822+0000", 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052855519999999998 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336875+0000", 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3137999999999998e-05 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.523860745, 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:29:50.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:29:50.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2761591960000001, 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:926 3.3 e43/30 3:c0b973a2:::rbd_data.1152dc9ca65d.0000000000000004:head v 43'30, mlcod=42'28)", 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2682283060000001, 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074617479999999998, 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586006+0000", 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.54e-07 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587534+0000", 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001528171 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587545+0000", 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0355e-05 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587551+0000", 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587615+0000", 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4119000000000001e-05 2024-09-17T23:29:50.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587631+0000", 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6181e-05 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593418+0000", 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057871729999999996 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593467+0000", 2024-09-17T23:29:50.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8945e-05 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:1015 3.16 e43/36 3:6edeb146:::rbd_data.1152dc9ca65d.0000000000000006:head v 43'86, mlcod=43'83)", 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2657005589999999, 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080340780000000001, 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:50.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:50.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588534+0000", 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.16e-07 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593477+0000", 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049433300000000001 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593490+0000", 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2874000000000001e-05 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593497+0000", 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1860000000000003e-06 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593548+0000", 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0964000000000002e-05 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593577+0000", 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9181000000000001e-05 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.596519+0000", 2024-09-17T23:29:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029411250000000002 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.596567+0000", 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8602e-05 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:50.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:50.971 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:50.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:50.973 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:50.974 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:50.974 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:50.969+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:50.981 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:50.981 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:50.997 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:50.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:50.998 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T23:29:50.998 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-17T23:29:50.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T23:29:50.999 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:50.999 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:51.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:29:51.018 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:51.018 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:51.038 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:51.038 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:51.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:51.059 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:51.093 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone14 from 0xe2bb000 (overlap 0xf59c00) to 0xdd0c00 2024-09-17T23:29:51.095 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.096 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:51.112 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:51.110+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:51.174 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:51.171+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:51.217 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.217 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:51.274 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:51.272+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:51.335 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.335 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:51.388 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:51.386+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:51.426 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T23:29:51.454 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.454 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.462 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:51.482 INFO:teuthology.orchestra.run.smithi139.stdout:1578 read 0x2033e00 thru 0x203e55e (0xa75f bytes) 2024-09-17T23:29:51.484 INFO:teuthology.orchestra.run.smithi139.stdout:1580 read 0x2c7da00 thru 0x2c85ec1 (0x84c2 bytes) 2024-09-17T23:29:51.485 INFO:teuthology.orchestra.run.smithi139.stdout:1584 punch from 0xde8c200 to 0xde8fc00, (0x3a00 bytes) 2024-09-17T23:29:51.496 INFO:teuthology.orchestra.run.smithi139.stdout:1585 write 0xdc97800 thru 0xdc97eb8 (0x6b9 bytes) 2024-09-17T23:29:51.497 INFO:teuthology.orchestra.run.smithi139.stdout:1588 clone 16 order 24 su 131072 sc 11 2024-09-17T23:29:51.504 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:51.502+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:51.576 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:51.574+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:51.609 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.610 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:51.677 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:51.675+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:51.725 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.726 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:51.778 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:51.775+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:51.841 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.841 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:51.879 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:51.876+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:51.956 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:51.956 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:51.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T23:29:52.001 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:29:52.009 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:52.007+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:52.076 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.077 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.081 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:52.079+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:52.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:52.094 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone15 intact 2024-09-17T23:29:52.182 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:52.180+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:52.283 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:52.281+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:52.312 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.312 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:52.384 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:52.382+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:52.430 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.431 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:52.484 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:52.482+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:52.546 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.547 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:52.585 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:52.584+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:52.687 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:52.685+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:52.708 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.708 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:52.745 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T23:29:52.787 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:52.785+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:52.834 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:52.834 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:52.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:52.865 INFO:teuthology.orchestra.run.smithi139.stdout:1591 write 0x29d0800 thru 0x29d82bc (0x7abd bytes) 2024-09-17T23:29:52.868 INFO:teuthology.orchestra.run.smithi139.stdout:1593 flatten 2024-09-17T23:29:52.928 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:52.925+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:52.989 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:52.987+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:53.017 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:53.018 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:53.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:53.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:53.090 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:53.088+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:53.191 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:53.188+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:53.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:53.293 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:53.293 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:53.294 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:53.290+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:53.331 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:53.366 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:53.393 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:53.391+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:53.397 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:53.414 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:53.414 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:53.439 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:53.439 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:53.456 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14717 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.418806754, 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007005636, 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14717 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034529+0000", 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034530+0000", 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.624e-06 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034535+0000", 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0170000000000004e-06 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034553+0000", 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7541e-05 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034567+0000", 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4446e-05 2024-09-17T23:29:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034630+0000", 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2283e-05 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034770+0000", 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014049000000000001 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041534+0000", 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067642350000000004 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14740 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.418748042000001, 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070617650000000002, 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14740 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034588+0000", 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9999999999999997e-07 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034592+0000", 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4010000000000001e-06 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034602+0000", 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0638e-05 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034608+0000", 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.241e-06 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035105+0000", 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049700000000000005 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035244+0000", 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013833000000000001 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041649+0000", 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064055550000000003 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14760 3.11 3:8ddf073a:::rbd_data.1152c1142907.0000000000000029:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.418713630999999, 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010811016999999999, 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14760 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034623+0000", 2024-09-17T23:29:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034627+0000", 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4680000000000003e-06 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034635+0000", 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034641+0000", 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035590+0000", 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094941100000000005 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036421+0000", 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083071299999999996 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036612+0000", 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191342 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041698+0000", 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050861980000000001 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045354+0000", 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036561440000000001 2024-09-17T23:29:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045394+0000", 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9604999999999998e-05 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045433+0000", 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8945000000000001e-05 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.417679696, 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:29:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:29:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:29:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.417506756, 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:29:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.417466886, 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:29:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:29:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14758 3.1 e41/33 3:853c0e15:::rbd_data.1152c1142907.0000000000000028:head v 41'194, mlcod=39'189)", 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.416102516, 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010716488999999999, 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037234+0000", 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5000000000000002e-07 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037239+0000", 2024-09-17T23:29:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.784e-06 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037249+0000", 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8889999999999997e-06 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037257+0000", 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.379e-06 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038255+0000", 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099873299999999991 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038288+0000", 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292000000000002e-05 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047928+0000", 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096399710000000007 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047950+0000", 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1790999999999999e-05 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14752 3.5 e41/36 3:a0bd3f31:::rbd_data.1152c1142907.0000000000000025:head v 41'34, mlcod=39'31)", 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.414248518000001, 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081097770000000003, 2024-09-17T23:29:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039096+0000", 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2230000000000007e-06 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039103+0000", 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7370000000000006e-06 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039109+0000", 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039160+0000", 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1072999999999997e-05 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039178+0000", 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8771e-05 2024-09-17T23:29:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047154+0000", 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079759010000000005 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047197+0000", 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2457999999999999e-05 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14838 3.3 e41/30 3:c39663f8:::rbd_data.1152c1142907.0000000000000064:head v 41'27, mlcod=39'22)", 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.410595056, 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071267700000000002, 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042741+0000", 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5300000000000003e-07 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042747+0000", 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-17T23:29:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042759+0000", 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2138e-05 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042766+0000", 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9979999999999997e-06 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042818+0000", 2024-09-17T23:29:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2191000000000002e-05 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042840+0000", 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1535e-05 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049823+0000", 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069831559999999999 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049867+0000", 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.439e-05 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14800 3.1 e41/33 3:8381b477:::rbd_data.1152c1142907.000000000000003d:head v 41'198, mlcod=39'189)", 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.408394699, 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007659412, 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044942+0000", 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-07 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044947+0000", 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.789e-06 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044958+0000", 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0223e-05 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044966+0000", 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999994e-06 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045312+0000", 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034605399999999999 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045344+0000", 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1999999999999999e-05 2024-09-17T23:29:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052554+0000", 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072101120000000003 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052600+0000", 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6521999999999999e-05 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21686 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.126155882999999, 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010211289, 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21686 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327182+0000", 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.199e-06 2024-09-17T23:29:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327654+0000", 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000472094 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327668+0000", 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3867e-05 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327676+0000", 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9289999999999998e-06 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327897+0000", 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022101 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327979+0000", 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1921000000000004e-05 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328181+0000", 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020200799999999999 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337270+0000", 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090895820000000006 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337286+0000", 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6112000000000001e-05 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337344+0000", 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7989999999999999e-05 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337391+0000", 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6576999999999998e-05 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21687 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.125634742000001, 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098426639999999992, 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21687 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5700000000000002e-07 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327704+0000", 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9349999999999999e-06 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327713+0000", 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-06 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327719+0000", 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9059999999999999e-06 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328700+0000", 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00098110899999999997 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328774+0000", 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3796999999999998e-05 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328937+0000", 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162934 2024-09-17T23:29:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337441+0000", 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085044609999999996 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337457+0000", 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6378000000000001e-05 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337503+0000", 2024-09-17T23:29:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5546000000000001e-05 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337543+0000", 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0559999999999998e-05 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21676 3.1 e41/33 3:87de3167:::rbd_data.1152c1142907.0000000000000151:head v 41'200, mlcod=41'196)", 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.125595102, 2024-09-17T23:29:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097022959999999991, 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327741+0000", 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327746+0000", 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327754+0000", 2024-09-17T23:29:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999992e-06 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327760+0000", 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327819+0000", 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9691000000000002e-05 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327838+0000", 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8691000000000001e-05 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337396+0000", 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095580470000000001 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337443+0000", 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6629999999999999e-05 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21694 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.124645565, 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090673060000000007, 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21694 2024-09-17T23:29:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328691+0000", 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328977+0000", 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028580700000000001 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328995+0000", 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329004+0000", 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3410000000000003e-06 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330527+0000", 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001522851 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330615+0000", 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8366000000000001e-05 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330857+0000", 2024-09-17T23:29:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024195999999999999 2024-09-17T23:29:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337600+0000", 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067435300000000002 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337662+0000", 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1246000000000003e-05 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337721+0000", 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8860000000000002e-05 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337757+0000", 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6749999999999999e-05 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21695 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.124143399999999, 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010154599, 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21695 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329193+0000", 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4499999999999995e-07 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329198+0000", 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329213+0000", 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4606999999999999e-05 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329220+0000", 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.328e-06 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331593+0000", 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002373287 2024-09-17T23:29:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331670+0000", 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6156999999999996e-05 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331868+0000", 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019856600000000001 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337649+0000", 2024-09-17T23:29:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057810889999999997 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339251+0000", 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016017309999999999 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339305+0000", 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3594000000000002e-05 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339347+0000", 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999997e-05 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21693 3.8 e41/30 3:12e31d2e:::rbd_data.1152c1142907.000000000000001a:head v 41'153, mlcod=41'146)", 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.12372762, 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072668539999999997, 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329609+0000", 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8899999999999999e-07 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329614+0000", 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5639999999999996e-06 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329623+0000", 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329633+0000", 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0003000000000001e-05 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331519+0000", 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018852509999999999 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331536+0000", 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7393000000000001e-05 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336822+0000", 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052855519999999998 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336875+0000", 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3137999999999998e-05 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.122962659000001, 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:29:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:29:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.8752611100000003, 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:926 3.3 e43/30 3:c0b973a2:::rbd_data.1152dc9ca65d.0000000000000004:head v 43'30, mlcod=42'28)", 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.8673302200000004, 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074617479999999998, 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586006+0000", 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.54e-07 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587534+0000", 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001528171 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587545+0000", 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0355e-05 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587551+0000", 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587615+0000", 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4119000000000001e-05 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587631+0000", 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6181e-05 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593418+0000", 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057871729999999996 2024-09-17T23:29:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593467+0000", 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8945e-05 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:1015 3.16 e43/36 3:6edeb146:::rbd_data.1152dc9ca65d.0000000000000006:head v 43'86, mlcod=43'83)", 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.8648024730000001, 2024-09-17T23:29:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080340780000000001, 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588534+0000", 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.16e-07 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593477+0000", 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049433300000000001 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593490+0000", 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2874000000000001e-05 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593497+0000", 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1860000000000003e-06 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593548+0000", 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0964000000000002e-05 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593577+0000", 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9181000000000001e-05 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.596519+0000", 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029411250000000002 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.596567+0000", 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8602e-05 2024-09-17T23:29:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:53.559 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:53.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:53.561 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:53.557+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:53.661 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:53.659+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:53.696 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:53.697 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:53.697 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:53.697 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:53.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:53.708 INFO:teuthology.orchestra.run.smithi139.stdout:1595 trunc from 0xe2bb000 to 0x6ff400 2024-09-17T23:29:53.754 INFO:teuthology.orchestra.run.smithi139.stdout:1598 read 0x55b200 thru 0x56288f (0x7690 bytes) 2024-09-17T23:29:53.756 INFO:teuthology.orchestra.run.smithi139.stdout:1599 punch from 0x214a00 to 0x219800, (0x4e00 bytes) 2024-09-17T23:29:53.756 INFO:teuthology.orchestra.run.smithi139.stdout:1601 punch from 0x19bc00 to 0x1a9600, (0xda00 bytes) 2024-09-17T23:29:53.756 INFO:teuthology.orchestra.run.smithi139.stdout:1605 read 0x38a800 thru 0x38f218 (0x4a19 bytes) 2024-09-17T23:29:53.756 INFO:teuthology.orchestra.run.smithi139.stdout:1606 punch from 0x5d2a00 to 0x5d7000, (0x4600 bytes) 2024-09-17T23:29:53.762 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:53.760+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:53.774 INFO:teuthology.orchestra.run.smithi139.stdout:1608 write 0x7865200 thru 0x786f4b7 (0xa2b8 bytes) 2024-09-17T23:29:53.790 INFO:teuthology.orchestra.run.smithi139.stdout:1609 write 0xaf51e00 thru 0xaf54c30 (0x2e31 bytes) 2024-09-17T23:29:53.792 INFO:teuthology.orchestra.run.smithi139.stdout:1613 write 0x233a00 thru 0x23e700 (0xad01 bytes) 2024-09-17T23:29:53.793 INFO:teuthology.orchestra.run.smithi139.stdout:1614 read 0x5ef8600 thru 0x5effb35 (0x7536 bytes) 2024-09-17T23:29:53.793 INFO:teuthology.orchestra.run.smithi139.stdout:1615 read 0xa1fa00 thru 0xa25b0e (0x610f bytes) 2024-09-17T23:29:53.795 INFO:teuthology.orchestra.run.smithi139.stdout:1616 trunc from 0xaf54e00 to 0xc9d3c00 2024-09-17T23:29:53.805 INFO:teuthology.orchestra.run.smithi139.stdout:1618 read 0x870ea00 thru 0x8712ac8 (0x40c9 bytes) 2024-09-17T23:29:53.807 INFO:teuthology.orchestra.run.smithi139.stdout:1621 punch from 0x34fde00 to 0x34fe000, (0x200 bytes) 2024-09-17T23:29:53.807 INFO:teuthology.orchestra.run.smithi139.stdout:1623 write 0x9e99c00 thru 0x9ea4d4a (0xb14b bytes) 2024-09-17T23:29:53.808 INFO:teuthology.orchestra.run.smithi139.stdout:1627 read 0x9471600 thru 0x9480773 (0xf174 bytes) 2024-09-17T23:29:53.808 INFO:teuthology.orchestra.run.smithi139.stdout:1628 read 0x3e16600 thru 0x3e22090 (0xba91 bytes) 2024-09-17T23:29:53.809 INFO:teuthology.orchestra.run.smithi139.stdout:1629 read 0x86acc00 thru 0x86b20ad (0x54ae bytes) 2024-09-17T23:29:53.809 INFO:teuthology.orchestra.run.smithi139.stdout:1633 read 0xc37fc00 thru 0xc386d65 (0x7166 bytes) 2024-09-17T23:29:53.810 INFO:teuthology.orchestra.run.smithi139.stdout:1635 trunc from 0xc9d3c00 to 0xd987400 2024-09-17T23:29:53.820 INFO:teuthology.orchestra.run.smithi139.stdout:1637 write 0x25c0800 thru 0x25cad6e (0xa56f bytes) 2024-09-17T23:29:53.821 INFO:teuthology.orchestra.run.smithi139.stdout:1638 read 0x56fd800 thru 0x56fdcc8 (0x4c9 bytes) 2024-09-17T23:29:53.831 INFO:teuthology.orchestra.run.smithi139.stdout:1640 trunc from 0xd987400 to 0x587d600 2024-09-17T23:29:53.852 INFO:teuthology.orchestra.run.smithi139.stdout:1641 trunc from 0x587d600 to 0x9c1ba00 2024-09-17T23:29:53.862 INFO:teuthology.orchestra.run.smithi139.stdout:1644 read 0x2b99800 thru 0x2ba9762 (0xff63 bytes) 2024-09-17T23:29:53.863 INFO:teuthology.orchestra.run.smithi139.stdout:1645 read 0x3feca00 thru 0x3ff4850 (0x7e51 bytes) 2024-09-17T23:29:53.863 INFO:teuthology.orchestra.run.smithi139.stdout:1648 read 0x8c3da00 thru 0x8c4ae72 (0xd473 bytes) 2024-09-17T23:29:53.864 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:53.861+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:53.864 INFO:teuthology.orchestra.run.smithi139.stdout:1649 punch from 0x7e2d600 to 0x7e3c400, (0xee00 bytes) 2024-09-17T23:29:53.864 INFO:teuthology.orchestra.run.smithi139.stdout:1652 write 0x600600 thru 0x60962c (0x902d bytes) 2024-09-17T23:29:53.864 INFO:teuthology.orchestra.run.smithi139.stdout:1657 punch from 0x767e000 to 0x7688200, (0xa200 bytes) 2024-09-17T23:29:53.865 INFO:teuthology.orchestra.run.smithi139.stdout:1658 read 0x5f83800 thru 0x5f8a744 (0x6f45 bytes) 2024-09-17T23:29:53.865 INFO:teuthology.orchestra.run.smithi139.stdout:1660 read 0x4ee4400 thru 0x4eea90a (0x650b bytes) 2024-09-17T23:29:53.867 INFO:teuthology.orchestra.run.smithi139.stdout:1661 read 0x87f1600 thru 0x87f1971 (0x372 bytes) 2024-09-17T23:29:53.867 INFO:teuthology.orchestra.run.smithi139.stdout:1662 read 0x41aac00 thru 0x41b14d2 (0x68d3 bytes) 2024-09-17T23:29:53.868 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:53.868 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:53.868 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:53.868 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:53.868 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1663 read 0x2217000 thru 0x221ab50 (0x3b51 bytes) 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1667 read 0x8977400 thru 0x89820a1 (0xaca2 bytes) 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1668 punch from 0x113aa00 to 0x1146000, (0xb600 bytes) 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1671 punch from 0x5dc5400 to 0x5dcf000, (0x9c00 bytes) 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1672 punch from 0x928800 to 0x92d400, (0x4c00 bytes) 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1673 punch from 0x7be5e00 to 0x7bf3c00, (0xde00 bytes) 2024-09-17T23:29:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:1674 read 0x5890000 thru 0x589c128 (0xc129 bytes) 2024-09-17T23:29:53.870 INFO:teuthology.orchestra.run.smithi139.stdout:1678 read 0x1d59a00 thru 0x1d5b8ac (0x1ead bytes) 2024-09-17T23:29:53.870 INFO:teuthology.orchestra.run.smithi139.stdout:1682 read 0x2fec000 thru 0x2ffbea2 (0xfea3 bytes) 2024-09-17T23:29:53.870 INFO:teuthology.orchestra.run.smithi139.stdout:1687 read 0x4459000 thru 0x44624af (0x94b0 bytes) 2024-09-17T23:29:53.871 INFO:teuthology.orchestra.run.smithi139.stdout:1688 write 0x8696400 thru 0x869d04d (0x6c4e bytes) 2024-09-17T23:29:53.872 INFO:teuthology.orchestra.run.smithi139.stdout:1690 write 0x1153c00 thru 0x11613c3 (0xd7c4 bytes) 2024-09-17T23:29:53.872 INFO:teuthology.orchestra.run.smithi139.stdout:1693 punch from 0xcaf800 to 0xcb7a00, (0x8200 bytes) 2024-09-17T23:29:53.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:53.883 INFO:teuthology.orchestra.run.smithi139.stdout:1694 write 0xad6a400 thru 0xad7228a (0x7e8b bytes) 2024-09-17T23:29:53.889 INFO:teuthology.orchestra.run.smithi139.stdout:1697 trunc from 0xad72400 to 0x74fa000 2024-09-17T23:29:53.903 INFO:teuthology.orchestra.run.smithi139.stdout:1698 punch from 0x1404800 to 0x1405200, (0xa00 bytes) 2024-09-17T23:29:53.925 INFO:teuthology.orchestra.run.smithi139.stdout:1699 write 0xdf0f400 thru 0xdf12544 (0x3145 bytes) 2024-09-17T23:29:53.925 INFO:teuthology.orchestra.run.smithi139.stdout:1700 read 0x8eb2600 thru 0x8ebf9de (0xd3df bytes) 2024-09-17T23:29:53.925 INFO:teuthology.orchestra.run.smithi139.stdout:1701 read 0x5aaea00 thru 0x5abe2ca (0xf8cb bytes) 2024-09-17T23:29:53.925 INFO:teuthology.orchestra.run.smithi139.stdout:1703 read 0x7a52200 thru 0x7a52a7d (0x87e bytes) 2024-09-17T23:29:53.942 INFO:teuthology.orchestra.run.smithi139.stdout:1705 trunc from 0xdf12600 to 0x28d000 2024-09-17T23:29:53.964 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:53.961+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:53.997 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.119989056000001, 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:29:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104363926000001, 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:29:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:29:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104326270000001, 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:29:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:29:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104308839000002, 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:29:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:29:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104273908, 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:29:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:29:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104242488000001, 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:29:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:29:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104225862, 2024-09-17T23:29:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:29:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104204167999999, 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:29:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:29:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.104192449999999, 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.539990106999999, 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:29:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:29:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.539967641, 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:29:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3262 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.532522862, 2024-09-17T23:29:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045783590999999998, 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3262 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463817+0000", 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00160411 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463821+0000", 2024-09-17T23:29:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6670000000000004e-06 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463838+0000", 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6898999999999999e-05 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463848+0000", 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507787+0000", 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043939260000000001 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507895+0000", 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010859899999999999 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507996+0000", 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100587 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.522629357, 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.520721462999999, 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:29:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.518736392999999, 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:29:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.518726797999999, 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:29:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:29:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12611 3.18 3:1c8edc2b:::rbd_data.115262cdab6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.48541607799999997, 2024-09-17T23:29:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.070059079999999996, 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12611 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509321+0000", 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-06 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553867+0000", 2024-09-17T23:29:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044545920000000003 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553893+0000", 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6094000000000001e-05 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553906+0000", 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3079e-05 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553972+0000", 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6630000000000004e-05 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.554090+0000", 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117364 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.555383+0000", 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001293188 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.577545+0000", 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.022161943 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579298+0000", 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017536349999999999 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579346+0000", 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7858000000000002e-05 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579378+0000", 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2011000000000002e-05 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12614 3.2 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.440795296, 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.100611516, 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12614 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553942+0000", 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.477e-06 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599428+0000", 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045486208 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599454+0000", 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6551999999999999e-05 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599467+0000", 2024-09-17T23:29:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2339e-05 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599548+0000", 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599660+0000", 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011209699999999999 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.601086+0000", 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014260550000000001 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.612776+0000", 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011690010000000001 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654410+0000", 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.041634517000000003 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654486+0000", 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6260999999999994e-05 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654552+0000", 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5086999999999995e-05 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4968.0:12613 3.9 e48/19 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head v 48'120, mlcod=45'117)", 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.39653720199999998, 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049712148999999997, 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598199+0000", 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-07 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637181+0000", 2024-09-17T23:29:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.038982528000000002 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637209+0000", 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7317e-05 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637219+0000", 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0372e-05 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637307+0000", 2024-09-17T23:29:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7867000000000004e-05 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637345+0000", 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7735999999999999e-05 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647845+0000", 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010500012 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647910+0000", 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5662999999999998e-05 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12676 3.1b 3:d8a99671:::rbd_data.115262cdab6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.390292588, 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.074765842999999998, 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12676 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:29:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604445+0000", 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1399999999999998e-06 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638570+0000", 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.034124814000000003 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638586+0000", 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6460000000000002e-05 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638611+0000", 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5216e-05 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638676+0000", 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5048999999999993e-05 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638792+0000", 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115233 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.640186+0000", 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013940090000000001 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657756+0000", 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017570240000000001 2024-09-17T23:29:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679079+0000", 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.021323312000000001 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679150+0000", 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1086000000000003e-05 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679209+0000", 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8284e-05 2024-09-17T23:29:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.103 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:54.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:29:54.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:54.106 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:54.103+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1706 write 0xe527200 thru 0xe529842 (0x2643 bytes) 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1711 trunc from 0xe529a00 to 0x56ff800 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1712 read 0x2edf200 thru 0x2eec665 (0xd466 bytes) 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1714 punch from 0x10fc600 to 0x1103000, (0x6a00 bytes) 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1716 trunc from 0x56ff800 to 0x5cc0a00 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1718 trunc from 0x5cc0a00 to 0x5826a00 2024-09-17T23:29:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:1719 read 0x433dc00 thru 0x43429c0 (0x4dc1 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1720 punch from 0xf1d800 to 0xf22200, (0x4a00 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1721 punch from 0x4be8a00 to 0x4bec400, (0x3a00 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1722 punch from 0x1f67c00 to 0x1f72200, (0xa600 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1727 read 0x1152800 thru 0x115c9f0 (0xa1f1 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1728 write 0xcbc200 thru 0xcc65bb (0xa3bc bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1729 read 0x4b84800 thru 0x4b8729b (0x2a9c bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1730 read 0x2e18600 thru 0x2e1c904 (0x4305 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1731 write 0x51ee200 thru 0x51f7eaf (0x9cb0 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1732 punch from 0x4a40200 to 0x4a44e00, (0x4c00 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1734 read 0x19ee600 thru 0x19eece6 (0x6e7 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1736 read 0xb12a00 thru 0xb15ceb (0x32ec bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1737 read 0x10b6c00 thru 0x10bfcbf (0x90c0 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1738 write 0xb9cfe00 thru 0xb9d4b7f (0x4d80 bytes) 2024-09-17T23:29:54.110 INFO:teuthology.orchestra.run.smithi139.stdout:1742 punch from 0x9073a00 to 0x9080e00, (0xd400 bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1743 write 0x9d5e600 thru 0x9d642a7 (0x5ca8 bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1745 write 0x6385800 thru 0x63893eb (0x3bec bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1747 read 0x2cb2e00 thru 0x2cb4fc9 (0x21ca bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1748 write 0x8f5ee00 thru 0x8f62f36 (0x4137 bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1749 read 0x952e600 thru 0x9538f4f (0xa950 bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1751 write 0x4ce5a00 thru 0x4cef4ad (0x9aae bytes) 2024-09-17T23:29:54.111 INFO:teuthology.orchestra.run.smithi139.stdout:1752 write 0x9861a00 thru 0x98663b5 (0x49b6 bytes) 2024-09-17T23:29:54.112 INFO:teuthology.orchestra.run.smithi139.stdout:1753 punch from 0x2eb4c00 to 0x2ec2c00, (0xe000 bytes) 2024-09-17T23:29:54.112 INFO:teuthology.orchestra.run.smithi139.stdout:1755 read 0x8fbd000 thru 0x8fcb771 (0xe772 bytes) 2024-09-17T23:29:54.130 INFO:teuthology.orchestra.run.smithi139.stdout:1756 write 0xc42fe00 thru 0xc43ee21 (0xf022 bytes) 2024-09-17T23:29:54.148 INFO:teuthology.orchestra.run.smithi139.stdout:1757 write 0xd7e0a00 thru 0xd7e29b3 (0x1fb4 bytes) 2024-09-17T23:29:54.158 INFO:teuthology.orchestra.run.smithi139.stdout:1759 trunc from 0xd7e2a00 to 0x4dc1800 2024-09-17T23:29:54.199 INFO:teuthology.orchestra.run.smithi139.stdout:1760 write 0x6426e00 thru 0x6435171 (0xe372 bytes) 2024-09-17T23:29:54.205 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:54.203+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:54.217 INFO:teuthology.orchestra.run.smithi139.stdout:1761 write 0xb7bd400 thru 0xb7c6700 (0x9301 bytes) 2024-09-17T23:29:54.217 INFO:teuthology.orchestra.run.smithi139.stdout:1762 read 0x968ae00 thru 0x9694f10 (0xa111 bytes) 2024-09-17T23:29:54.218 INFO:teuthology.orchestra.run.smithi139.stdout:1766 read 0x1125200 thru 0x1135158 (0xff59 bytes) 2024-09-17T23:29:54.238 INFO:teuthology.orchestra.run.smithi139.stdout:1767 write 0xedac200 thru 0xedb228d (0x608e bytes) 2024-09-17T23:29:54.239 INFO:teuthology.orchestra.run.smithi139.stdout:1769 write 0x9de9000 thru 0x9dea6fb (0x16fc bytes) 2024-09-17T23:29:54.249 INFO:teuthology.orchestra.run.smithi139.stdout:1771 trunc from 0xedb2400 to 0x69c0e00 2024-09-17T23:29:54.267 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:54.267 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:54.267 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:54.268 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:54.270 INFO:teuthology.orchestra.run.smithi139.stdout:1773 read 0x5515400 thru 0x5519f1c (0x4b1d bytes) 2024-09-17T23:29:54.275 INFO:teuthology.orchestra.run.smithi139.stdout:1774 trunc from 0x69c0e00 to 0x943a600 2024-09-17T23:29:54.277 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:54.284 INFO:teuthology.orchestra.run.smithi139.stdout:1779 read 0x337e800 thru 0x3388e61 (0xa662 bytes) 2024-09-17T23:29:54.299 INFO:teuthology.orchestra.run.smithi139.stdout:1782 write 0xc5f2600 thru 0xc5fd6f3 (0xb0f4 bytes) 2024-09-17T23:29:54.299 INFO:teuthology.orchestra.run.smithi139.stdout:1788 punch from 0x3a59400 to 0x3a5fc00, (0x6800 bytes) 2024-09-17T23:29:54.300 INFO:teuthology.orchestra.run.smithi139.stdout:1789 punch from 0x47c7c00 to 0x47d2a00, (0xae00 bytes) 2024-09-17T23:29:54.300 INFO:teuthology.orchestra.run.smithi139.stdout:1790 read 0x7579000 thru 0x7587ee9 (0xeeea bytes) 2024-09-17T23:29:54.301 INFO:teuthology.orchestra.run.smithi139.stdout:1791 read 0x76cc00 thru 0x770dc8 (0x41c9 bytes) 2024-09-17T23:29:54.302 INFO:teuthology.orchestra.run.smithi139.stdout:1793 read 0x3253e00 thru 0x325d6cd (0x98ce bytes) 2024-09-17T23:29:54.302 INFO:teuthology.orchestra.run.smithi139.stdout:1794 read 0x1876a00 thru 0x18771d2 (0x7d3 bytes) 2024-09-17T23:29:54.302 INFO:teuthology.orchestra.run.smithi139.stdout:1795 write 0x61a8600 thru 0x61b5337 (0xcd38 bytes) 2024-09-17T23:29:54.303 INFO:teuthology.orchestra.run.smithi139.stdout:1796 read 0x3bfc800 thru 0x3c051b2 (0x89b3 bytes) 2024-09-17T23:29:54.304 INFO:teuthology.orchestra.run.smithi139.stdout:1797 read 0x5c16200 thru 0x5c253e6 (0xf1e7 bytes) 2024-09-17T23:29:54.304 INFO:teuthology.orchestra.run.smithi139.stdout:1800 read 0xc03a000 thru 0xc04344f (0x9450 bytes) 2024-09-17T23:29:54.304 INFO:teuthology.orchestra.run.smithi139.stdout:1804 punch from 0xaf1f600 to 0xaf24800, (0x5200 bytes) 2024-09-17T23:29:54.306 INFO:teuthology.orchestra.run.smithi139.stdout:1806 trunc from 0xc5fd800 to 0xd949000 2024-09-17T23:29:54.316 INFO:teuthology.orchestra.run.smithi139.stdout:1808 punch from 0x2b94e00 to 0x2ba1600, (0xc800 bytes) 2024-09-17T23:29:54.317 INFO:teuthology.orchestra.run.smithi139.stdout:1809 read 0x857ec00 thru 0x8586b2e (0x7f2f bytes) 2024-09-17T23:29:54.317 INFO:teuthology.orchestra.run.smithi139.stdout:1811 read 0x1656600 thru 0x16618a3 (0xb2a4 bytes) 2024-09-17T23:29:54.320 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:54.318+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:54.329 INFO:teuthology.orchestra.run.smithi139.stdout:1814 trunc from 0xd949000 to 0x3c61400 2024-09-17T23:29:54.353 INFO:teuthology.orchestra.run.smithi139.stdout:1817 read 0xda2600 thru 0xdb0445 (0xde46 bytes) 2024-09-17T23:29:54.354 INFO:teuthology.orchestra.run.smithi139.stdout:1820 write 0x2fb9400 thru 0x2fbf4f4 (0x60f5 bytes) 2024-09-17T23:29:54.381 INFO:teuthology.orchestra.run.smithi139.stdout:1821 write 0xdc39000 thru 0xdc44184 (0xb185 bytes) 2024-09-17T23:29:54.382 INFO:teuthology.orchestra.run.smithi139.stdout:1822 read 0x94d0600 thru 0x94d098c (0x38d bytes) 2024-09-17T23:29:54.382 INFO:teuthology.orchestra.run.smithi139.stdout:1826 read 0xb3e6800 thru 0xb3f443a (0xdc3b bytes) 2024-09-17T23:29:54.382 INFO:teuthology.orchestra.run.smithi139.stdout:1827 read 0x969a600 thru 0x96a8d92 (0xe793 bytes) 2024-09-17T23:29:54.386 INFO:teuthology.orchestra.run.smithi139.stdout:1830 read 0x5787a00 thru 0x5793c4a (0xc24b bytes) 2024-09-17T23:29:54.392 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:54.392 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:54.392 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:54.392 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:54.392 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:54.395 INFO:teuthology.orchestra.run.smithi139.stdout:1833 trunc from 0xdc44200 to 0x60b7000 2024-09-17T23:29:54.401 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:54.408 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:54.405+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:54.420 INFO:teuthology.orchestra.run.smithi139.stdout:1834 read 0x225f600 thru 0x2263235 (0x3c36 bytes) 2024-09-17T23:29:54.427 INFO:teuthology.orchestra.run.smithi139.stdout:1835 trunc from 0x60b7000 to 0x667c800 2024-09-17T23:29:54.456 INFO:teuthology.orchestra.run.smithi139.stdout:1836 write 0xe9d4000 thru 0xe9de1ec (0xa1ed bytes) 2024-09-17T23:29:54.457 INFO:teuthology.orchestra.run.smithi139.stdout:1837 punch from 0xc84b200 to 0xc857400, (0xc200 bytes) 2024-09-17T23:29:54.457 INFO:teuthology.orchestra.run.smithi139.stdout:1838 read 0x6d5f800 thru 0x6d6274b (0x2f4c bytes) 2024-09-17T23:29:54.458 INFO:teuthology.orchestra.run.smithi139.stdout:1839 read 0x642e200 thru 0x64326cc (0x44cd bytes) 2024-09-17T23:29:54.458 INFO:teuthology.orchestra.run.smithi139.stdout:1840 write 0x906fc00 thru 0x907963a (0x9a3b bytes) 2024-09-17T23:29:54.459 INFO:teuthology.orchestra.run.smithi139.stdout:1841 read 0x5ef4800 thru 0x5eff422 (0xac23 bytes) 2024-09-17T23:29:54.459 INFO:teuthology.orchestra.run.smithi139.stdout:1842 write 0x195c600 thru 0x19698e8 (0xd2e9 bytes) 2024-09-17T23:29:54.461 INFO:teuthology.orchestra.run.smithi139.stdout:1843 read 0xe0bfe00 thru 0xe0c808e (0x828f bytes) 2024-09-17T23:29:54.461 INFO:teuthology.orchestra.run.smithi139.stdout:1845 write 0x492600 thru 0x49ef6e (0xc96f bytes) 2024-09-17T23:29:54.462 INFO:teuthology.orchestra.run.smithi139.stdout:1847 write 0xc7c800 thru 0xc7cfcb (0x7cc bytes) 2024-09-17T23:29:54.462 INFO:teuthology.orchestra.run.smithi139.stdout:1849 write 0x75e2c00 thru 0x75efd69 (0xd16a bytes) 2024-09-17T23:29:54.463 INFO:teuthology.orchestra.run.smithi139.stdout:1850 read 0x42d7200 thru 0x42e03db (0x91dc bytes) 2024-09-17T23:29:54.464 INFO:teuthology.orchestra.run.smithi139.stdout:1852 read 0xd2df800 thru 0xd2dfe78 (0x679 bytes) 2024-09-17T23:29:54.464 INFO:teuthology.orchestra.run.smithi139.stdout:1853 read 0xc195600 thru 0xc1a0705 (0xb106 bytes) 2024-09-17T23:29:54.464 INFO:teuthology.orchestra.run.smithi139.stdout:1854 read 0xd524e00 thru 0xd52e81d (0x9a1e bytes) 2024-09-17T23:29:54.465 INFO:teuthology.orchestra.run.smithi139.stdout:1855 punch from 0x4c64200 to 0x4c68000, (0x3e00 bytes) 2024-09-17T23:29:54.465 INFO:teuthology.orchestra.run.smithi139.stdout:1857 write 0xc489600 thru 0xc48df99 (0x499a bytes) 2024-09-17T23:29:54.465 INFO:teuthology.orchestra.run.smithi139.stdout:1859 write 0xcec5800 thru 0xced011f (0xa920 bytes) 2024-09-17T23:29:54.465 INFO:teuthology.orchestra.run.smithi139.stdout:1864 write 0xba01600 thru 0xba0ddd5 (0xc7d6 bytes) 2024-09-17T23:29:54.478 INFO:teuthology.orchestra.run.smithi139.stdout:1867 trunc from 0xe9de200 to 0x3357c00 2024-09-17T23:29:54.506 INFO:teuthology.orchestra.run.smithi139.stdout:1868 trunc from 0x3357c00 to 0x38e4200 2024-09-17T23:29:54.509 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:54.506+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:54.530 INFO:teuthology.orchestra.run.smithi139.stdout:1870 trunc from 0x38e4200 to 0xe627e00 2024-09-17T23:29:54.540 INFO:teuthology.orchestra.run.smithi139.stdout:1871 read 0x739e400 thru 0x739f4c1 (0x10c2 bytes) 2024-09-17T23:29:54.553 INFO:teuthology.orchestra.run.smithi139.stdout:1873 trunc from 0xe627e00 to 0x4a3f000 2024-09-17T23:29:54.564 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1983 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1805312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.068703297999999, 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048765663000000001, 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1983 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492091+0000", 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492092+0000", 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.22e-06 2024-09-17T23:29:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492095+0000", 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2279999999999998e-06 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492098+0000", 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540732+0000", 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048634183999999997 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540781+0000", 2024-09-17T23:29:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8976e-05 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540856+0000", 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5142000000000007e-05 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1984 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1814528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.068692681999998, 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049003441000000002, 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1984 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492103+0000", 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1540000000000001e-06 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492107+0000", 2024-09-17T23:29:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0249999999999996e-06 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492109+0000", 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540902+0000", 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048793087999999998 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540995+0000", 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2534000000000005e-05 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541105+0000", 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000109815 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1985 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1823744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.068681388000002, 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049142263999999998, 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1985 2024-09-17T23:29:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493825+0000", 2024-09-17T23:29:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017121879999999999 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493829+0000", 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2509999999999996e-06 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493836+0000", 2024-09-17T23:29:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000002e-06 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493840+0000", 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3220000000000003e-06 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541141+0000", 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047301395000000003 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541190+0000", 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8819999999999997e-05 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541255+0000", 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4587999999999999e-05 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2106 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1096704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.06684143, 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797818999999999, 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2106 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493953+0000", 2024-09-17T23:29:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1699999999999998e-07 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493955+0000", 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4059999999999999e-06 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493962+0000", 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6379999999999998e-06 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493968+0000", 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4149999999999998e-06 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542562+0000", 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048594796000000003 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542650+0000", 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7288999999999997e-05 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542750+0000", 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100758 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.587 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2107 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1105920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.066819223, 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048932067000000003, 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2107 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.22e-07 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493977+0000", 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493984+0000", 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0920000000000003e-06 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493989+0000", 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0370000000000002e-06 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542786+0000", 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797034000000003 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542841+0000", 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5781e-05 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542907+0000", 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5362000000000002e-05 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2108 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1115136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.066798692999999, 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049148982000000001, 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2108 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493996+0000", 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.03e-07 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493998+0000", 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.187e-06 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494004+0000", 2024-09-17T23:29:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2219999999999999e-06 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494009+0000", 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6870000000000002e-06 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542954+0000", 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048945206999999998 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543043+0000", 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9494999999999996e-05 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543144+0000", 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100781 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.066778703000001, 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:29:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.066758823000001, 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:29:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:29:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.066737284999999, 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:29:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2117 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1198080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064874970000002, 2024-09-17T23:29:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048875710000000003, 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2117 2024-09-17T23:29:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:29:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495920+0000", 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.105e-06 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495924+0000", 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.269e-06 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495926+0000", 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5830000000000001e-06 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544618+0000", 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048691940000000003 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544704+0000", 2024-09-17T23:29:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5581000000000006e-05 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544795+0000", 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1020999999999995e-05 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2118 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1207296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064864762999999, 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.04902596, 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2118 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495931+0000", 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.139e-06 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495934+0000", 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1870000000000001e-06 2024-09-17T23:29:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495936+0000", 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.542e-06 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544842+0000", 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048905954000000001 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544893+0000", 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0927999999999998e-05 2024-09-17T23:29:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544955+0000", 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999000000000001e-05 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064854687, 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:29:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064844570999998, 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:29:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:29:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064832340999999, 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:29:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:29:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064821771999998, 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:29:54.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:29:54.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.648 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064811884000001, 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.649 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:29:54.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:29:54.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.655 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064801601999999, 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:29:54.656 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:29:54.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:29:54.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.661 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.064785714999999, 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:29:54.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:29:54.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2128 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1299456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.666 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.062900873, 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048685818999999998, 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2128 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497893+0000", 2024-09-17T23:29:54.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5800000000000001e-07 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497895+0000", 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.44e-06 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497905+0000", 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0231000000000001e-05 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497912+0000", 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8970000000000002e-06 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546386+0000", 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048474585000000001 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546477+0000", 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0301000000000002e-05 2024-09-17T23:29:54.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546579+0000", 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102107 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2129 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1308672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.062877803999999, 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048890621000000002, 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:54.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2129 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5100000000000001e-07 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497918+0000", 2024-09-17T23:29:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4670000000000001e-06 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497922+0000", 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:54.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497924+0000", 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6390000000000001e-06 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546622+0000", 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048697427000000001 2024-09-17T23:29:54.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546706+0000", 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4306000000000002e-05 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:54.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:54.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546807+0000", 2024-09-17T23:29:54.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100851 2024-09-17T23:29:54.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.677 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:54.678 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:54.678 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:54.680 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:54.676+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:54.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:54.684 INFO:teuthology.orchestra.run.smithi139.stdout:1874 read 0x3b25600 thru 0x3b2ec34 (0x9635 bytes) 2024-09-17T23:29:54.684 INFO:teuthology.orchestra.run.smithi139.stdout:1876 write 0x34e5600 thru 0x34f0dab (0xb7ac bytes) 2024-09-17T23:29:54.684 INFO:teuthology.orchestra.run.smithi139.stdout:1878 punch from 0x3647e00 to 0x364c200, (0x4400 bytes) 2024-09-17T23:29:54.684 INFO:teuthology.orchestra.run.smithi139.stdout:1880 read 0x46d1600 thru 0x46df17f (0xdb80 bytes) 2024-09-17T23:29:54.684 INFO:teuthology.orchestra.run.smithi139.stdout:1881 write 0x1b73600 thru 0x1b7a767 (0x7168 bytes) 2024-09-17T23:29:54.685 INFO:teuthology.orchestra.run.smithi139.stdout:1883 read 0x42d3800 thru 0x42da735 (0x6f36 bytes) 2024-09-17T23:29:54.685 INFO:teuthology.orchestra.run.smithi139.stdout:1886 write 0xbf78e00 thru 0xbf7939d (0x59e bytes) 2024-09-17T23:29:54.685 INFO:teuthology.orchestra.run.smithi139.stdout:1890 write 0x92ede00 thru 0x92fa1de (0xc3df bytes) 2024-09-17T23:29:54.685 INFO:teuthology.orchestra.run.smithi139.stdout:1892 read 0x993f800 thru 0x994c542 (0xcd43 bytes) 2024-09-17T23:29:54.685 INFO:teuthology.orchestra.run.smithi139.stdout:1893 trunc from 0xbf79400 to 0xcfe1e00 2024-09-17T23:29:54.685 INFO:teuthology.orchestra.run.smithi139.stdout:1894 write 0x4250000 thru 0x425bd07 (0xbd08 bytes) 2024-09-17T23:29:54.686 INFO:teuthology.orchestra.run.smithi139.stdout:1895 write 0xebe1a00 thru 0xebe45de (0x2bdf bytes) 2024-09-17T23:29:54.686 INFO:teuthology.orchestra.run.smithi139.stdout:1897 punch from 0xa4e5000 to 0xa4eda00, (0x8a00 bytes) 2024-09-17T23:29:54.686 INFO:teuthology.orchestra.run.smithi139.stdout:1898 punch from 0xe4c6c00 to 0xe4cbe00, (0x5200 bytes) 2024-09-17T23:29:54.686 INFO:teuthology.orchestra.run.smithi139.stdout:1900 clone 17 order 22 su 32768 sc 13 2024-09-17T23:29:54.780 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:54.778+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:54.793 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:54.793 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:54.793 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:29:54.793 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:54.802 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:54.881 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:54.879+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:54.909 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:54.909 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:54.909 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:54.909 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:54.909 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:54.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:54.983 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:54.980+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:55.016 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:29:55.069 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:55.069 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1357 3.17 3:eddf8349:::rbd_data.1152ac6484f7.0000000000000006:10 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3164752440000003, 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025710974000000001, 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1357 2024-09-17T23:29:55.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749700+0000", 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749702+0000", 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9879999999999999e-06 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749708+0000", 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0730000000000001e-06 2024-09-17T23:29:55.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749713+0000", 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1250000000000001e-06 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775212+0000", 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025499457 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775293+0000", 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0961e-05 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775410+0000", 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000116896 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1372 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:55.074 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3164547229999997, 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025690635999999999, 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1372 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7899999999999999e-07 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749722+0000", 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1560000000000002e-06 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749729+0000", 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749734+0000", 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8450000000000002e-06 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775202+0000", 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025468409000000001 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775294+0000", 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1453999999999995e-05 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775410+0000", 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000116964 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1373 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3164326319999997, 2024-09-17T23:29:55.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025821230000000001, 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1373 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4000000000000002e-07 2024-09-17T23:29:55.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749744+0000", 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0729999999999999e-06 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749751+0000", 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4740000000000002e-06 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749756+0000", 2024-09-17T23:29:55.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0719999999999999e-06 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775440+0000", 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025684489000000001 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775490+0000", 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9629999999999997e-05 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775563+0000", 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3052000000000005e-05 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1374 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 167936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3164117309999996, 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026046814000000001, 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1374 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749765+0000", 2024-09-17T23:29:55.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0719999999999998e-06 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749772+0000", 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5690000000000001e-06 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749782+0000", 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7489999999999993e-06 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775609+0000", 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025827815 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775699+0000", 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9202000000000004e-05 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775810+0000", 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011099699999999999 2024-09-17T23:29:55.088 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1375 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3163862450000003, 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.02628755, 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.089 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1375 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:29:55.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749789+0000", 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.84e-07 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749790+0000", 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5889999999999999e-06 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749794+0000", 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8310000000000003e-06 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749797+0000", 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6369999999999999e-06 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775863+0000", 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026065878000000001 2024-09-17T23:29:55.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775958+0000", 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5678000000000001e-05 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776076+0000", 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011765300000000001 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3163718309999997, 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:55.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3163394689999999, 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:29:55.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:29:55.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3163182869999996, 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.103 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3162917250000001, 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:29:55.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:29:55.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3162695519999996, 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:29:55.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:29:55.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.116 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3162496399999997, 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:29:55.120 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3162029640000004, 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.121 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:55.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3161762159999997, 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:29:55.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:29:55.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.316148128, 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:29:55.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.316127829, 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.135 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:29:55.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1744 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3100572230000003, 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025572786, 2024-09-17T23:29:55.139 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1744 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756118+0000", 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756120+0000", 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0260000000000002e-06 2024-09-17T23:29:55.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756124+0000", 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5340000000000001e-06 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756128+0000", 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.811e-06 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781601+0000", 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025472859 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781649+0000", 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7641e-05 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781690+0000", 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1467000000000001e-05 2024-09-17T23:29:55.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1745 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 823296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3100396349999999, 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025663729, 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.144 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1745 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9800000000000004e-07 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756138+0000", 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1519999999999999e-06 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756143+0000", 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8769999999999998e-06 2024-09-17T23:29:55.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756148+0000", 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5630000000000002e-06 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781715+0000", 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025566720000000001 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781758+0000", 2024-09-17T23:29:55.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3683000000000002e-05 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781799+0000", 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0235999999999999e-05 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1746 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 905216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:29:55.148 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3100133600000001, 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025754597000000001, 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1746 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:29:55.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756162+0000", 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7400000000000003e-07 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756164+0000", 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.407e-06 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756169+0000", 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0030000000000002e-06 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756173+0000", 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9589999999999997e-06 2024-09-17T23:29:55.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781831+0000", 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025658045000000001 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781875+0000", 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4280000000000003e-05 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781916+0000", 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0329e-05 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1747 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.3099965579999999, 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025889276999999999, 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:55.153 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1747 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7000000000000001e-07 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756180+0000", 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3540000000000001e-06 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756183+0000", 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3400000000000002e-06 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756185+0000", 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.486e-06 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781947+0000", 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025761954 2024-09-17T23:29:55.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.782018+0000", 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0278999999999996e-05 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.782067+0000", 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9594e-05 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.489329302, 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:55.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:29:55.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:29:55.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:55.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:55.163 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:55.165 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:55.163+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:55.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:55.265 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:55.262+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:55.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:55.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:55.279 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:55.280 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:55.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:55.366 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:55.364+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:55.443 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:55.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:55.443 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:55.443 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:55.443 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:55.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:29:55.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:55.467 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:55.465+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:55.491 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T23:29:55.568 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:55.566+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9550 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3300043069999998, 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056192745000000002, 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9550 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290751+0000", 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290757+0000", 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290762+0000", 2024-09-17T23:29:55.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346584+0000", 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055821636000000001 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346864+0000", 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028079899999999998 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346941+0000", 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6173000000000004e-05 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9552 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.329767393, 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056146769999999999, 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9552 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8999999999999996e-07 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290994+0000", 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3399999999999998e-06 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291001+0000", 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1910000000000003e-06 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291007+0000", 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346974+0000", 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055966828000000003 2024-09-17T23:29:55.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347059+0000", 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5753000000000004e-05 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347131+0000", 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9604 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3296980199999999, 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05626722, 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9604 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291055+0000", 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291057+0000", 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291064+0000", 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291070+0000", 2024-09-17T23:29:55.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347164+0000", 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056094887000000003 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347250+0000", 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5552999999999999e-05 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347321+0000", 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1310999999999995e-05 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9606 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3294323280000002, 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05619619, 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9606 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1900000000000002e-07 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291324+0000", 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6619999999999998e-06 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291333+0000", 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291339+0000", 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.477e-06 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347352+0000", 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056012987 2024-09-17T23:29:55.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347434+0000", 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1569000000000005e-05 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347516+0000", 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2365000000000002e-05 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9608 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.329402167, 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056951664999999999, 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9608 2024-09-17T23:29:55.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291351+0000", 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8299999999999997e-07 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291353+0000", 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.396e-06 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291360+0000", 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.199e-06 2024-09-17T23:29:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291367+0000", 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2230000000000001e-06 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347546+0000", 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056178616000000001 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348232+0000", 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068600200000000001 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348302+0000", 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9646000000000004e-05 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9610 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:55.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3292394769999998, 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056957865000000003, 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9610 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291517+0000", 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.393e-06 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291526+0000", 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0129999999999999e-06 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291532+0000", 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8430000000000004e-06 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348331+0000", 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056798279 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348405+0000", 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4418000000000005e-05 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348470+0000", 2024-09-17T23:29:55.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5355e-05 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9613 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3291694330000001, 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057056663000000001, 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9613 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:55.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291585+0000", 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291593+0000", 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291596+0000", 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.269e-06 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348498+0000", 2024-09-17T23:29:55.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056902369000000001 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348573+0000", 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4962000000000005e-05 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348639+0000", 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5874e-05 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9615 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3290057150000001, 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057044395999999997, 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9615 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291747+0000", 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291749+0000", 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291754+0000", 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410000000000003e-06 2024-09-17T23:29:55.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291757+0000", 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4520000000000002e-06 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348665+0000", 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056908513000000001 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348732+0000", 2024-09-17T23:29:55.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6180999999999995e-05 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348791+0000", 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9129000000000003e-05 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9620 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.328851362, 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057063968999999999, 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9620 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:29:55.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291909+0000", 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5610000000000001e-06 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291916+0000", 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0679999999999999e-06 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291923+0000", 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348817+0000", 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056893959000000001 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348898+0000", 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0377999999999995e-05 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348965+0000", 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7146999999999999e-05 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9622 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3287928870000001, 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057231199000000003, 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9622 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291960+0000", 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291962+0000", 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7939999999999998e-06 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291970+0000", 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5009999999999998e-06 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291975+0000", 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999999e-06 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349015+0000", 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057039443000000002 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349104+0000", 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9791999999999997e-05 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349190+0000", 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6055000000000002e-05 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9716 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.324646988, 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056085545000000001, 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9716 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296106+0000", 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296108+0000", 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296114+0000", 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7830000000000004e-06 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296119+0000", 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352074+0000", 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055955619999999998 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352121+0000", 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6857999999999998e-05 2024-09-17T23:29:55.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352191+0000", 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9639000000000002e-05 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9722 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3245880699999999, 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056209310999999998, 2024-09-17T23:29:55.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9722 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296362+0000", 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019768800000000001 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296366+0000", 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9419999999999997e-06 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296375+0000", 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6309999999999998e-06 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296392+0000", 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6758000000000001e-05 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352221+0000", 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055828950000000002 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352299+0000", 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8375999999999996e-05 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352373+0000", 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3966000000000003e-05 2024-09-17T23:29:55.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9723 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3243491619999999, 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056145054999999999, 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9723 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296404+0000", 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8800000000000002e-07 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296411+0000", 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296419+0000", 2024-09-17T23:29:55.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129999999999993e-06 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296424+0000", 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5879999999999997e-06 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352405+0000", 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055980769 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352479+0000", 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4018999999999997e-05 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352548+0000", 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8997000000000004e-05 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9729 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:29:55.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3242939809999998, 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056272003000000001, 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9729 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296459+0000", 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2799999999999996e-07 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296461+0000", 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439999999999998e-06 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296468+0000", 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296472+0000", 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-17T23:29:55.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352583+0000", 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056111017999999999 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352665+0000", 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1148000000000003e-05 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352730+0000", 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5477999999999999e-05 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9731 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3242725960000001, 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056506531999999998, 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9731 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297569+0000", 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001089394 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297574+0000", 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7260000000000002e-06 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297583+0000", 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4169999999999998e-06 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297590+0000", 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352775+0000", 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055185161000000003 2024-09-17T23:29:55.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352867+0000", 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2090999999999996e-05 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352986+0000", 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011883300000000001 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9743 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3230029820000002, 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055972380000000002, 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9743 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297750+0000", 2024-09-17T23:29:55.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297752+0000", 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297760+0000", 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1839999999999999e-06 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297766+0000", 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353551+0000", 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055784783999999997 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353645+0000", 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3760999999999999e-05 2024-09-17T23:29:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353721+0000", 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6907000000000001e-05 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9744 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3229781140000001, 2024-09-17T23:29:55.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056155391999999998, 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9744 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297775+0000", 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297777+0000", 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298431+0000", 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065376599999999996 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298441+0000", 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8709999999999992e-06 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353773+0000", 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055332332999999997 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353856+0000", 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3122999999999997e-05 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353929+0000", 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3183999999999996e-05 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9795 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 4677632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3205942889999998, 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056138625999999997, 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9795 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301567+0000", 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001409492 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301573+0000", 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2159999999999997e-06 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301582+0000", 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5619999999999993e-06 2024-09-17T23:29:55.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301588+0000", 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356166+0000", 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054577696000000002 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356253+0000", 2024-09-17T23:29:55.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6526000000000005e-05 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356296+0000", 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3918999999999997e-05 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12612 3.13 3:ccf171c8:::rbd_data.115262cdab6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.1100610359999998, 2024-09-17T23:29:55.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063461453000000001, 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12612 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510693+0000", 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9e-06 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539114+0000", 2024-09-17T23:29:55.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028420697000000002 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539145+0000", 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1488000000000001e-05 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539159+0000", 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3737000000000001e-05 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539226+0000", 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7572999999999998e-05 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539337+0000", 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110849 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.540857+0000", 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001520043 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:55.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.571640+0000", 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030782961000000001 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574080+0000", 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024394529999999998 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574120+0000", 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0459e-05 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574153+0000", 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292999999999997e-05 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12613 3.9 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.0815630340000002, 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.13090919500000001, 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:55.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12613 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539191+0000", 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.59e-06 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.595998+0000", 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056807745999999999 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596021+0000", 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2574000000000001e-05 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596030+0000", 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1670000000000005e-06 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596150+0000", 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119688 2024-09-17T23:29:55.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596301+0000", 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015097400000000001 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598103+0000", 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001802281 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.610230+0000", 2024-09-17T23:29:55.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012126468 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.669969+0000", 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059739318999999999 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670041+0000", 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1834000000000004e-05 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:55.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670098+0000", 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7553999999999998e-05 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:55.719 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:55.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:55.721 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:55.721 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:55.722 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:55.722 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:55.723 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:55.723 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:55.717+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:55.733 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:55.733 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:55.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:55.757 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:55.821 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:55.819+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:55.876 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:55.877 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:55.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:55.922 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:55.920+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:56.023 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:56.020+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:56.031 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:56.031 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:56.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:56.124 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:56.122+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:56.225 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:56.222+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:56.228 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:56.228 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:56.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:56.326 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:56.324+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:56.419 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:56.419 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:56.427 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:56.425+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:56.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:56.528 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:56.525+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:56.573 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:56.573 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:56.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:56.628 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:56.626+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:56.729 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:56.727+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:56.765 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:56.765 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:56.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:56.830 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:56.828+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:56.931 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:56.931 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:56.931 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:56.929+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:56.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:29:57.020 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:29:57.020 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:29:57.020 INFO:tasks.thrashosds.thrasher:Reweighting osd 5 to 0.4953324759108587 2024-09-17T23:29:57.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 0.4953324759108587 2024-09-17T23:29:57.031 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:57.029+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:57.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:29:57.098 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:57.098 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:57.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:29:57.133 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:57.130+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:57.222 INFO:teuthology.orchestra.run.smithi139.stdout:1901 write 0x685a600 thru 0x685bad8 (0x14d9 bytes) 2024-09-17T23:29:57.223 INFO:teuthology.orchestra.run.smithi139.stdout:1902 write 0x26dd000 thru 0x26ec59e (0xf59f bytes) 2024-09-17T23:29:57.224 INFO:teuthology.orchestra.run.smithi139.stdout:1905 write 0x1c5400 thru 0x1d0a3f (0xb640 bytes) 2024-09-17T23:29:57.228 INFO:teuthology.orchestra.run.smithi139.stdout:1906 trunc from 0xebe4600 to 0xc28fc00 2024-09-17T23:29:57.236 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:57.233+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:57.272 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:57.272 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:57.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:29:57.365 INFO:teuthology.orchestra.run.smithi139.stdout:1908 write 0x6bee800 thru 0x6bf3776 (0x4f77 bytes) 2024-09-17T23:29:57.366 INFO:teuthology.orchestra.run.smithi139.stdout:1910 read 0x77dea00 thru 0x77ec2da (0xd8db bytes) 2024-09-17T23:29:57.366 INFO:teuthology.orchestra.run.smithi139.stdout:1911 punch from 0x47c9600 to 0x47cd200, (0x3c00 bytes) 2024-09-17T23:29:57.367 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:57.365+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:57.369 INFO:teuthology.orchestra.run.smithi139.stdout:1914 write 0x989f000 thru 0x989ffe6 (0xfe7 bytes) 2024-09-17T23:29:57.375 INFO:teuthology.orchestra.run.smithi139.stdout:1917 write 0xaad6000 thru 0xaadd98c (0x798d bytes) 2024-09-17T23:29:57.376 INFO:teuthology.orchestra.run.smithi139.stdout:1921 punch from 0x9a8ee00 to 0x9a95000, (0x6200 bytes) 2024-09-17T23:29:57.406 INFO:teuthology.orchestra.run.smithi139.stdout:1923 write 0xd0ba000 thru 0xd0bd157 (0x3158 bytes) 2024-09-17T23:29:57.406 INFO:teuthology.orchestra.run.smithi139.stdout:1924 write 0x53ce00 thru 0x545610 (0x8811 bytes) 2024-09-17T23:29:57.422 INFO:teuthology.orchestra.run.smithi139.stdout:1925 trunc from 0xd0bd200 to 0x54ba00 2024-09-17T23:29:57.436 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:57.433+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:57.471 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:57.472 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:57.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:29:57.505 INFO:teuthology.orchestra.run.smithi139.stdout:1928 trunc from 0x54ba00 to 0xda18200 2024-09-17T23:29:57.526 INFO:teuthology.orchestra.run.smithi139.stdout:1930 trunc from 0xda18200 to 0x3bbaa00 2024-09-17T23:29:57.537 INFO:teuthology.orchestra.run.smithi139.stdout:1932 punch from 0x273600 to 0x27ac00, (0x7600 bytes) 2024-09-17T23:29:57.538 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:57.534+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:57.557 INFO:teuthology.orchestra.run.smithi139.stdout:1933 trunc from 0x3bbaa00 to 0xd53c000 2024-09-17T23:29:57.566 INFO:teuthology.orchestra.run.smithi139.stdout:1934 punch from 0x607cc00 to 0x6086800, (0x9c00 bytes) 2024-09-17T23:29:57.572 INFO:teuthology.orchestra.run.smithi139.stdout:1935 read 0xc450600 thru 0xc45bf0f (0xb910 bytes) 2024-09-17T23:29:57.586 INFO:teuthology.orchestra.run.smithi139.stdout:1938 write 0xd616a00 thru 0xd6237c3 (0xcdc4 bytes) 2024-09-17T23:29:57.597 INFO:teuthology.orchestra.run.smithi139.stdout:1939 trunc from 0xd623800 to 0x3f63600 2024-09-17T23:29:57.621 INFO:teuthology.orchestra.run.smithi139.stdout:1941 read 0xbc9400 thru 0xbd3878 (0xa479 bytes) 2024-09-17T23:29:57.626 INFO:teuthology.orchestra.run.smithi139.stdout:1944 read 0x3478800 thru 0x347bd46 (0x3547 bytes) 2024-09-17T23:29:57.632 INFO:teuthology.orchestra.run.smithi139.stdout:1947 trunc from 0x3f63600 to 0x8da7200 2024-09-17T23:29:57.639 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:57.635+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:57.645 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:57.645 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:57.655 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:29:57.660 INFO:teuthology.orchestra.run.smithi139.stdout:1948 write 0xae03a00 thru 0xae11658 (0xdc59 bytes) 2024-09-17T23:29:57.660 INFO:teuthology.orchestra.run.smithi139.stdout:1950 write 0x8cdca00 thru 0x8ceba6d (0xf06e bytes) 2024-09-17T23:29:57.660 INFO:teuthology.orchestra.run.smithi139.stdout:1952 write 0x6b3fe00 thru 0x6b46de2 (0x6fe3 bytes) 2024-09-17T23:29:57.660 INFO:teuthology.orchestra.run.smithi139.stdout:1955 read 0xf01600 thru 0xf0399a (0x239b bytes) 2024-09-17T23:29:57.690 INFO:teuthology.orchestra.run.smithi139.stdout:1956 write 0xdf98800 thru 0xdfa65b9 (0xddba bytes) 2024-09-17T23:29:57.712 INFO:teuthology.orchestra.run.smithi139.stdout:1957 write 0xe30b200 thru 0xe317b20 (0xc921 bytes) 2024-09-17T23:29:57.712 INFO:teuthology.orchestra.run.smithi139.stdout:1958 write 0x42e3800 thru 0x42e6032 (0x2833 bytes) 2024-09-17T23:29:57.712 INFO:teuthology.orchestra.run.smithi139.stdout:1959 write 0x8a6c400 thru 0x8a6d753 (0x1354 bytes) 2024-09-17T23:29:57.713 INFO:teuthology.orchestra.run.smithi139.stdout:1960 read 0x42ea00 thru 0x43506c (0x666d bytes) 2024-09-17T23:29:57.713 INFO:teuthology.orchestra.run.smithi139.stdout:1962 clone 18 order 22 su 32768 sc 12 2024-09-17T23:29:57.739 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:57.737+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:57.769 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:57.769 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:57.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:29:57.777 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:29:57.864 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:57.862+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:57.895 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:29:57.895 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:29:57.912 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:29:57.912 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:29:57.925 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:29:57.926 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:29:57.928 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:29:57.929 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:29:57.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:29:57.945 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:29:57.945 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:29:57.962 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:29:57.963 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:29:57.976 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:57.974+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:57.979 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:29:57.980 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:29:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 0.495332 (7ece) 2024-09-17T23:29:58.019 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:29:58.082 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:58.079+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:58.087 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:29:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:29:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:29:58.087 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:58.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:29:58.142 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:29:58.140+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:29:58.204 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:29:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1357 3.17 3:eddf8349:::rbd_data.1152ac6484f7.0000000000000006:10 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4519034220000009, 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025710974000000001, 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1357 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749700+0000", 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749702+0000", 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9879999999999999e-06 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749708+0000", 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0730000000000001e-06 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749713+0000", 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1250000000000001e-06 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775212+0000", 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025499457 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775293+0000", 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0961e-05 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775410+0000", 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000116896 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1372 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4518829009999994, 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025690635999999999, 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1372 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7899999999999999e-07 2024-09-17T23:29:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749722+0000", 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1560000000000002e-06 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749729+0000", 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749734+0000", 2024-09-17T23:29:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8450000000000002e-06 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775202+0000", 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025468409000000001 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775294+0000", 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1453999999999995e-05 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775410+0000", 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000116964 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1373 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4518608099999994, 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025821230000000001, 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1373 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4000000000000002e-07 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749744+0000", 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0729999999999999e-06 2024-09-17T23:29:58.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749751+0000", 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4740000000000002e-06 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749756+0000", 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0719999999999999e-06 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775440+0000", 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025684489000000001 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775490+0000", 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9629999999999997e-05 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775563+0000", 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3052000000000005e-05 2024-09-17T23:29:58.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1374 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 167936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4518399090000003, 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026046814000000001, 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.219 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1374 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:58.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749765+0000", 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0719999999999998e-06 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749772+0000", 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5690000000000001e-06 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749782+0000", 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7489999999999993e-06 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775609+0000", 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025827815 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775699+0000", 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9202000000000004e-05 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775810+0000", 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011099699999999999 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1375 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4518144230000001, 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.02628755, 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1375 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.224 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749789+0000", 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.84e-07 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749790+0000", 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5889999999999999e-06 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749794+0000", 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8310000000000003e-06 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749797+0000", 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6369999999999999e-06 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775863+0000", 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026065878000000001 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775958+0000", 2024-09-17T23:29:58.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5678000000000001e-05 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776076+0000", 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011765300000000001 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:58.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4518000089999994, 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:29:58.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4517676470000005, 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:29:58.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:29:58.237 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4517464649999994, 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:29:58.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:58.242 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4517199030000008, 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:58.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.246 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4516977299999994, 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:29:58.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:29:58.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4516778180000003, 2024-09-17T23:29:58.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:29:58.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:29:58.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4516311420000001, 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:29:58.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:29:58.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4516043940000003, 2024-09-17T23:29:58.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4515763059999998, 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:29:58.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4515560070000006, 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:29:58.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1744 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4454854009999991, 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025572786, 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1744 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.275 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756118+0000", 2024-09-17T23:29:58.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756120+0000", 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0260000000000002e-06 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756124+0000", 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5340000000000001e-06 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756128+0000", 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.811e-06 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781601+0000", 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025472859 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781649+0000", 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7641e-05 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781690+0000", 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1467000000000001e-05 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1745 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 823296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4454678130000005, 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025663729, 2024-09-17T23:29:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1745 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9800000000000004e-07 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756138+0000", 2024-09-17T23:29:58.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1519999999999999e-06 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756143+0000", 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8769999999999998e-06 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756148+0000", 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5630000000000002e-06 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781715+0000", 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025566720000000001 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781758+0000", 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3683000000000002e-05 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781799+0000", 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0235999999999999e-05 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1746 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 905216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4454415380000007, 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025754597000000001, 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1746 2024-09-17T23:29:58.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756162+0000", 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7400000000000003e-07 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756164+0000", 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.407e-06 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756169+0000", 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0030000000000002e-06 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756173+0000", 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9589999999999997e-06 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781831+0000", 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025658045000000001 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781875+0000", 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4280000000000003e-05 2024-09-17T23:29:58.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781916+0000", 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0329e-05 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1747 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4454247359999997, 2024-09-17T23:29:58.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025889276999999999, 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1747 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:29:58.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7000000000000001e-07 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756180+0000", 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3540000000000001e-06 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756183+0000", 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3400000000000002e-06 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756185+0000", 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.486e-06 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781947+0000", 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025761954 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.782018+0000", 2024-09-17T23:29:58.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0278999999999996e-05 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.782067+0000", 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9594e-05 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6247574799999995, 2024-09-17T23:29:58.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:29:58.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:29:58.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:29:58.299 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:29:58.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:29:58.302 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:58.299+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:58.402 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:58.399+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:58.411 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:58.411 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:58.411 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:58.411 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:58.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:29:58.503 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:58.501+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:58.568 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:58.568 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:58.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:29:58.618 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:58.615+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:58.704 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:58.702+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:58.726 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:58.726 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:58.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:58.726 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:58.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9550 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4324400649999998, 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056192745000000002, 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9550 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290751+0000", 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290757+0000", 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:29:58.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290762+0000", 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346584+0000", 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055821636000000001 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346864+0000", 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028079899999999998 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346941+0000", 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6173000000000004e-05 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9552 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4322031510000004, 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056146769999999999, 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9552 2024-09-17T23:29:58.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8999999999999996e-07 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290994+0000", 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3399999999999998e-06 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291001+0000", 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1910000000000003e-06 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291007+0000", 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346974+0000", 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055966828000000003 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347059+0000", 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5753000000000004e-05 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347131+0000", 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9604 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4321337779999999, 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05626722, 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9604 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291055+0000", 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291057+0000", 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:29:58.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291064+0000", 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291070+0000", 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347164+0000", 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056094887000000003 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347250+0000", 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5552999999999999e-05 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347321+0000", 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1310999999999995e-05 2024-09-17T23:29:58.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9606 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4318680859999997, 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05619619, 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9606 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1900000000000002e-07 2024-09-17T23:29:58.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291324+0000", 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6619999999999998e-06 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291333+0000", 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291339+0000", 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.477e-06 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347352+0000", 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056012987 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347434+0000", 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1569000000000005e-05 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347516+0000", 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2365000000000002e-05 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9608 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.431837925, 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056951664999999999, 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9608 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291351+0000", 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8299999999999997e-07 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291353+0000", 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.396e-06 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291360+0000", 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.199e-06 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291367+0000", 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2230000000000001e-06 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347546+0000", 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056178616000000001 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348232+0000", 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068600200000000001 2024-09-17T23:29:58.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348302+0000", 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9646000000000004e-05 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9610 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4316752350000002, 2024-09-17T23:29:58.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056957865000000003, 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9610 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:29:58.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291517+0000", 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.393e-06 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291526+0000", 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0129999999999999e-06 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291532+0000", 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8430000000000004e-06 2024-09-17T23:29:58.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348331+0000", 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056798279 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348405+0000", 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4418000000000005e-05 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348470+0000", 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5355e-05 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9613 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4316051910000001, 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057056663000000001, 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9613 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291585+0000", 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291593+0000", 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291596+0000", 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.269e-06 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348498+0000", 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056902369000000001 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348573+0000", 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4962000000000005e-05 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348639+0000", 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5874e-05 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9615 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4314414729999996, 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057044395999999997, 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9615 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291747+0000", 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:29:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291749+0000", 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291754+0000", 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410000000000003e-06 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291757+0000", 2024-09-17T23:29:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4520000000000002e-06 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348665+0000", 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056908513000000001 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348732+0000", 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6180999999999995e-05 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348791+0000", 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9129000000000003e-05 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9620 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4312871200000004, 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057063968999999999, 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9620 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:58.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291909+0000", 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5610000000000001e-06 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291916+0000", 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0679999999999999e-06 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291923+0000", 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348817+0000", 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056893959000000001 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348898+0000", 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0377999999999995e-05 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348965+0000", 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7146999999999999e-05 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9622 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:29:58.771 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.431228645, 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057231199000000003, 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9622 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:29:58.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291960+0000", 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291962+0000", 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7939999999999998e-06 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291970+0000", 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5009999999999998e-06 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291975+0000", 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999999e-06 2024-09-17T23:29:58.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349015+0000", 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057039443000000002 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349104+0000", 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9791999999999997e-05 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349190+0000", 2024-09-17T23:29:58.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6055000000000002e-05 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9716 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.427082746, 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056085545000000001, 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9716 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296106+0000", 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296108+0000", 2024-09-17T23:29:58.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296114+0000", 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7830000000000004e-06 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296119+0000", 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352074+0000", 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055955619999999998 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352121+0000", 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6857999999999998e-05 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352191+0000", 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9639000000000002e-05 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.780 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9722 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4270238280000003, 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056209310999999998, 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9722 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296362+0000", 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019768800000000001 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296366+0000", 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9419999999999997e-06 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296375+0000", 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6309999999999998e-06 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296392+0000", 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6758000000000001e-05 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352221+0000", 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055828950000000002 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352299+0000", 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8375999999999996e-05 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352373+0000", 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3966000000000003e-05 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9723 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4267849200000002, 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056145054999999999, 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9723 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296404+0000", 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8800000000000002e-07 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296411+0000", 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296419+0000", 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129999999999993e-06 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296424+0000", 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5879999999999997e-06 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352405+0000", 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055980769 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352479+0000", 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4018999999999997e-05 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352548+0000", 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8997000000000004e-05 2024-09-17T23:29:58.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9729 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4267297389999998, 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056272003000000001, 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9729 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296459+0000", 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2799999999999996e-07 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296461+0000", 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439999999999998e-06 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296468+0000", 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-17T23:29:58.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296472+0000", 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352583+0000", 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056111017999999999 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352665+0000", 2024-09-17T23:29:58.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1148000000000003e-05 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352730+0000", 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5477999999999999e-05 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9731 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4267083539999996, 2024-09-17T23:29:58.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056506531999999998, 2024-09-17T23:29:58.795 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9731 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297569+0000", 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001089394 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297574+0000", 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7260000000000002e-06 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297583+0000", 2024-09-17T23:29:58.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4169999999999998e-06 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297590+0000", 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352775+0000", 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055185161000000003 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352867+0000", 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2090999999999996e-05 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352986+0000", 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011883300000000001 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9743 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:29:58.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4254387399999997, 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055972380000000002, 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9743 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:29:58.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297750+0000", 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297752+0000", 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297760+0000", 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1839999999999999e-06 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297766+0000", 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353551+0000", 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055784783999999997 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353645+0000", 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3760999999999999e-05 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353721+0000", 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6907000000000001e-05 2024-09-17T23:29:58.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9744 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.425413872, 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056155391999999998, 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9744 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297775+0000", 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297777+0000", 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298431+0000", 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065376599999999996 2024-09-17T23:29:58.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298441+0000", 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8709999999999992e-06 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353773+0000", 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055332332999999997 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353856+0000", 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3122999999999997e-05 2024-09-17T23:29:58.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353929+0000", 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3183999999999996e-05 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9795 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 4677632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4230300470000001, 2024-09-17T23:29:58.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056138625999999997, 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9795 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301567+0000", 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001409492 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301573+0000", 2024-09-17T23:29:58.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2159999999999997e-06 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301582+0000", 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5619999999999993e-06 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301588+0000", 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356166+0000", 2024-09-17T23:29:58.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054577696000000002 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356253+0000", 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6526000000000005e-05 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356296+0000", 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3918999999999997e-05 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12612 3.13 3:ccf171c8:::rbd_data.115262cdab6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2124967939999998, 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063461453000000001, 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12612 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510693+0000", 2024-09-17T23:29:58.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9e-06 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539114+0000", 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028420697000000002 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539145+0000", 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1488000000000001e-05 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539159+0000", 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3737000000000001e-05 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539226+0000", 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7572999999999998e-05 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539337+0000", 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110849 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.540857+0000", 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001520043 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.571640+0000", 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030782961000000001 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574080+0000", 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024394529999999998 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574120+0000", 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0459e-05 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574153+0000", 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292999999999997e-05 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12613 3.9 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1839987919999997, 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.13090919500000001, 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12613 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539191+0000", 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.59e-06 2024-09-17T23:29:58.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.595998+0000", 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056807745999999999 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596021+0000", 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2574000000000001e-05 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596030+0000", 2024-09-17T23:29:58.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1670000000000005e-06 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596150+0000", 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119688 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596301+0000", 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015097400000000001 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:29:58.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598103+0000", 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001802281 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.610230+0000", 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012126468 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.669969+0000", 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059739318999999999 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:58.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670041+0000", 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1834000000000004e-05 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670098+0000", 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7553999999999998e-05 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:58.824 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:58.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:29:58.826 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:58.823+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:58.927 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:29:58.924+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:29:58.978 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:58.978 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:58.978 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:58.978 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:58.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:29:59.029 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:59.026+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:59.109 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone17 from 0xe317c00 (overlap 0x54ba00) to 0x32d600 2024-09-17T23:29:59.128 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:59.126+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:59.159 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:59.159 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:59.159 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:59.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:29:59.229 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:29:59.227+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.804368355000001, 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:29:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9465325419999999, 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9458402489999997, 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9458286679999999, 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:29:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:29:59.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:29:59.301 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9453014209999999, 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:29:59.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9452674380000001, 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:29:59.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:29:59.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9449568460000002, 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:29:59.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9449307119999997, 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9444328049999999, 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:29:59.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.943831715, 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:29:59.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9432830800000005, 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:29:59.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9430754820000002, 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:29:59.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:29:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.337 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9334595810000001, 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9334493469999998, 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:29:59.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:29:59.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:29:59.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9334394799999997, 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:29:59.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9334293320000002, 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:29:59.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:29:59.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9329594639999996, 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:29:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:29:59.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9329373480000003, 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:29:59.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9326496530000004, 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:29:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:29:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7472634830000002, 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:29:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:29:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:29:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.375 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:59.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:29:59.377 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:59.374+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:59.437 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T23:29:59.478 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:29:59.475+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:29:59.486 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:59.486 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:29:59.486 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:59.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:29:59.578 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:59.576+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:59.673 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:59.673 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:29:59.673 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:29:59.673 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:29:59.673 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:59.681 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:29:59.679+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:29:59.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:29:59.782 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:59.780+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14760 3.11 3:8ddf073a:::rbd_data.1152c1142907.0000000000000029:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:59.812 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.773883340000001, 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010811016999999999, 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14760 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034623+0000", 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034627+0000", 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4680000000000003e-06 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034635+0000", 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-17T23:29:59.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034641+0000", 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035590+0000", 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094941100000000005 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036421+0000", 2024-09-17T23:29:59.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083071299999999996 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036612+0000", 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191342 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041698+0000", 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050861980000000001 2024-09-17T23:29:59.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045354+0000", 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036561440000000001 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045394+0000", 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9604999999999998e-05 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045433+0000", 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8945000000000001e-05 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.772849404999999, 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:29:59.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:29:59.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:29:59.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.772676465, 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:29:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:29:59.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.772636595000002, 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:29:59.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.838 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14758 3.1 e41/33 3:853c0e15:::rbd_data.1152c1142907.0000000000000028:head v 41'194, mlcod=39'189)", 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.771272225000001, 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010716488999999999, 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037234+0000", 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5000000000000002e-07 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037239+0000", 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.784e-06 2024-09-17T23:29:59.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037249+0000", 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8889999999999997e-06 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037257+0000", 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.379e-06 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038255+0000", 2024-09-17T23:29:59.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099873299999999991 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038288+0000", 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292000000000002e-05 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047928+0000", 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096399710000000007 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047950+0000", 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1790999999999999e-05 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14752 3.5 e41/36 3:a0bd3f31:::rbd_data.1152c1142907.0000000000000025:head v 41'34, mlcod=39'31)", 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.769418226999999, 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081097770000000003, 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.843 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039087+0000", 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.58e-07 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039096+0000", 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2230000000000007e-06 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039103+0000", 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7370000000000006e-06 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039109+0000", 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039160+0000", 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1072999999999997e-05 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.039178+0000", 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8771e-05 2024-09-17T23:29:59.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047154+0000", 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079759010000000005 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047197+0000", 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2457999999999999e-05 2024-09-17T23:29:59.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14800 3.1 e41/33 3:8381b477:::rbd_data.1152c1142907.000000000000003d:head v 41'198, mlcod=39'189)", 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.763564408000001, 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007659412, 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044941+0000", 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044942+0000", 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-07 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044947+0000", 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.789e-06 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044958+0000", 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0223e-05 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.044966+0000", 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0299999999999994e-06 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045312+0000", 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034605399999999999 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045344+0000", 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1999999999999999e-05 2024-09-17T23:29:59.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052554+0000", 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072101120000000003 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.052600+0000", 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6521999999999999e-05 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21686 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.481325592000001, 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010211289, 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21686 2024-09-17T23:29:59.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327182+0000", 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.199e-06 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327654+0000", 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000472094 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327668+0000", 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3867e-05 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327676+0000", 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9289999999999998e-06 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327897+0000", 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022101 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327979+0000", 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1921000000000004e-05 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328181+0000", 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020200799999999999 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337270+0000", 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090895820000000006 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337286+0000", 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6112000000000001e-05 2024-09-17T23:29:59.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337344+0000", 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7989999999999999e-05 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337391+0000", 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6576999999999998e-05 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21687 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.480804451000001, 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098426639999999992, 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21687 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.859 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327701+0000", 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5700000000000002e-07 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327704+0000", 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9349999999999999e-06 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327713+0000", 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3809999999999999e-06 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327719+0000", 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9059999999999999e-06 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328700+0000", 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00098110899999999997 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328774+0000", 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3796999999999998e-05 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328937+0000", 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000162934 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337441+0000", 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085044609999999996 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337457+0000", 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6378000000000001e-05 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337503+0000", 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5546000000000001e-05 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337543+0000", 2024-09-17T23:29:59.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0559999999999998e-05 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21676 3.1 e41/33 3:87de3167:::rbd_data.1152c1142907.0000000000000151:head v 41'200, mlcod=41'196)", 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.480764811, 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097022959999999991, 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327740+0000", 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327741+0000", 2024-09-17T23:29:59.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327746+0000", 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327754+0000", 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999992e-06 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327760+0000", 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327819+0000", 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9691000000000002e-05 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327838+0000", 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8691000000000001e-05 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337396+0000", 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095580470000000001 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337443+0000", 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6629999999999999e-05 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21694 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.479815274, 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090673060000000007, 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21694 2024-09-17T23:29:59.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328690+0000", 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328691+0000", 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328977+0000", 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028580700000000001 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328995+0000", 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8428999999999999e-05 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329004+0000", 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3410000000000003e-06 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330527+0000", 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001522851 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330615+0000", 2024-09-17T23:29:59.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8366000000000001e-05 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330857+0000", 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024195999999999999 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337600+0000", 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067435300000000002 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337662+0000", 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1246000000000003e-05 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337721+0000", 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8860000000000002e-05 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337757+0000", 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6749999999999999e-05 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21695 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.479313109, 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010154599, 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21695 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329193+0000", 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4499999999999995e-07 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329198+0000", 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329213+0000", 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4606999999999999e-05 2024-09-17T23:29:59.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329220+0000", 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.328e-06 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331593+0000", 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002373287 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331670+0000", 2024-09-17T23:29:59.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6156999999999996e-05 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331868+0000", 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019856600000000001 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337649+0000", 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057810889999999997 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339251+0000", 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016017309999999999 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339305+0000", 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3594000000000002e-05 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339347+0000", 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999997e-05 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:21693 3.8 e41/30 3:12e31d2e:::rbd_data.1152c1142907.000000000000001a:head v 41'153, mlcod=41'146)", 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.478897328999999, 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072668539999999997, 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:59.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329608+0000", 2024-09-17T23:29:59.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329609+0000", 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8899999999999999e-07 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329614+0000", 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5639999999999996e-06 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329623+0000", 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329633+0000", 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0003000000000001e-05 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331519+0000", 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018852509999999999 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331536+0000", 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7393000000000001e-05 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336822+0000", 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052855519999999998 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.336875+0000", 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3137999999999998e-05 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.478132368000001, 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:29:59.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:29:59.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:29:59.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:29:59.894 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.230430819, 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:29:59.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:926 3.3 e43/30 3:c0b973a2:::rbd_data.1152dc9ca65d.0000000000000004:head v 43'30, mlcod=42'28)", 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.222499929, 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074617479999999998, 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586005+0000", 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.586006+0000", 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.54e-07 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587534+0000", 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001528171 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587545+0000", 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0355e-05 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587551+0000", 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0499999999999997e-06 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587615+0000", 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4119000000000001e-05 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587631+0000", 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6181e-05 2024-09-17T23:29:59.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593418+0000", 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057871729999999996 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593467+0000", 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8945e-05 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:1015 3.16 e43/36 3:6edeb146:::rbd_data.1152dc9ca65d.0000000000000006:head v 43'86, mlcod=43'83)", 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.219972181999999, 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080340780000000001, 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:59.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588533+0000", 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.588534+0000", 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.16e-07 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593477+0000", 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049433300000000001 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593490+0000", 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2874000000000001e-05 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593497+0000", 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1860000000000003e-06 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593548+0000", 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0964000000000002e-05 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593577+0000", 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9181000000000001e-05 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.596519+0000", 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029411250000000002 2024-09-17T23:29:59.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.596567+0000", 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8602e-05 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.2204985529999997, 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:29:59.908 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:29:59.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:29:59.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.2072715040000004, 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:29:59.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:29:59.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5001.0:4523 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:head [stat,truncate 262144] snapc 14=[14] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:59.131914+0000", 2024-09-17T23:29:59.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.67659113199999998, 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075071199999999999, 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5001", 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4523 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:29:59.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131914+0000", 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131914+0000", 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131915+0000", 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.361e-06 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131922+0000", 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0039999999999996e-06 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131933+0000", 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1195e-05 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131938+0000", 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.113e-06 2024-09-17T23:29:59.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.131993+0000", 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5362000000000003e-05 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.132043+0000", 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9697000000000002e-05 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.132142+0000", 2024-09-17T23:29:59.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9529000000000002e-05 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.136919+0000", 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047769520000000001 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.139322+0000", 2024-09-17T23:29:59.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024030829999999999 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.139379+0000", 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6437999999999997e-05 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:59.139421+0000", 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2385999999999997e-05 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:29:59.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:29:59.924 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:29:59.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:29:59.926 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:29:59.923+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:29:59.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:00.033 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:00.033 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:00.033 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:00.033 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:00.034 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:00.032+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:00.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:30:00.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:00.119 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:00.127 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:00.125+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:00.132 INFO:teuthology.orchestra.run.smithi139.stdout:1963 write 0x49a0000 thru 0x49a813d (0x813e bytes) 2024-09-17T23:30:00.132 INFO:teuthology.orchestra.run.smithi139.stdout:1966 read 0xade5c00 thru 0xadeebe5 (0x8fe6 bytes) 2024-09-17T23:30:00.133 INFO:teuthology.orchestra.run.smithi139.stdout:1967 write 0x8c43a00 thru 0x8c51c51 (0xe252 bytes) 2024-09-17T23:30:00.135 INFO:teuthology.orchestra.run.smithi139.stdout:1968 punch from 0x9c31c00 to 0x9c3e200, (0xc600 bytes) 2024-09-17T23:30:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:00.138 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:00.154 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:00.154 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:00.154 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:00.154 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:00.154 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:00.166 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:00.168 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:00.168 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:00.170 INFO:teuthology.orchestra.run.smithi139.stdout:1969 read 0x4af6a00 thru 0x4b02430 (0xba31 bytes) 2024-09-17T23:30:00.184 INFO:teuthology.orchestra.run.smithi139.stdout:1970 trunc from 0xe317c00 to 0x3387800 2024-09-17T23:30:00.192 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:00.193 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:00.224 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:00.224 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:00.227 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:00.225+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:00.239 INFO:teuthology.orchestra.run.smithi139.stdout:1971 read 0x85fa00 thru 0x860829 (0xe2a bytes) 2024-09-17T23:30:00.240 INFO:teuthology.orchestra.run.smithi139.stdout:1976 read 0x2fdaa00 thru 0x2fe2c77 (0x8278 bytes) 2024-09-17T23:30:00.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:00.245 INFO:teuthology.orchestra.run.smithi139.stdout:1978 trunc from 0x3387800 to 0x6d23a00 2024-09-17T23:30:00.254 INFO:teuthology.orchestra.run.smithi139.stdout:1979 write 0x4f9f600 thru 0x4fa01fd (0xbfe bytes) 2024-09-17T23:30:00.256 INFO:teuthology.orchestra.run.smithi139.stdout:1984 write 0xa8800 thru 0xb4cd2 (0xc4d3 bytes) 2024-09-17T23:30:00.283 INFO:teuthology.orchestra.run.smithi139.stdout:1985 write 0x8f1e200 thru 0x8f1f7c2 (0x15c3 bytes) 2024-09-17T23:30:00.284 INFO:teuthology.orchestra.run.smithi139.stdout:1986 trunc from 0x8f1f800 to 0x80bca00 2024-09-17T23:30:00.299 INFO:teuthology.orchestra.run.smithi139.stdout:1990 flatten 2024-09-17T23:30:00.329 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:00.326+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:00.347 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:00.347 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:00.357 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:00.379 INFO:teuthology.orchestra.run.smithi139.stdout:1991 read 0x436e800 thru 0x4372d42 (0x4543 bytes) 2024-09-17T23:30:00.380 INFO:teuthology.orchestra.run.smithi139.stdout:1992 read 0x1c73c00 thru 0x1c7f186 (0xb587 bytes) 2024-09-17T23:30:00.390 INFO:teuthology.orchestra.run.smithi139.stdout:1993 write 0x8669e00 thru 0x867515d (0xb35e bytes) 2024-09-17T23:30:00.396 INFO:teuthology.orchestra.run.smithi139.stdout:1994 read 0x816d000 thru 0x816d0f0 (0xf1 bytes) 2024-09-17T23:30:00.417 INFO:teuthology.orchestra.run.smithi139.stdout:1995 write 0x9e8e000 thru 0x9e9c7d3 (0xe7d4 bytes) 2024-09-17T23:30:00.418 INFO:teuthology.orchestra.run.smithi139.stdout:1996 read 0x58b8e00 thru 0x58bf1fe (0x63ff bytes) 2024-09-17T23:30:00.418 INFO:teuthology.orchestra.run.smithi139.stdout:1998 write 0x162e000 thru 0x1638a8d (0xaa8e bytes) 2024-09-17T23:30:00.418 INFO:teuthology.orchestra.run.smithi139.stdout:1999 read 0x59b9000 thru 0x59c4b5c (0xbb5d bytes) 2024-09-17T23:30:00.418 INFO:teuthology.orchestra.run.smithi139.stdout:2000 write 0x6b47800 thru 0x6b51b94 (0xa395 bytes) 2024-09-17T23:30:00.423 INFO:teuthology.orchestra.run.smithi139.stdout:2001 trunc from 0x9e9c800 to 0xe3ac800 2024-09-17T23:30:00.429 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:00.428+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:00.446 INFO:teuthology.orchestra.run.smithi139.stdout:2002 punch from 0x127a200 to 0x1283c00, (0x9a00 bytes) 2024-09-17T23:30:00.446 INFO:teuthology.orchestra.run.smithi139.stdout:2003 punch from 0x544ce00 to 0x5458e00, (0xc000 bytes) 2024-09-17T23:30:00.446 INFO:teuthology.orchestra.run.smithi139.stdout:2005 read 0xa710400 thru 0xa719e4d (0x9a4e bytes) 2024-09-17T23:30:00.446 INFO:teuthology.orchestra.run.smithi139.stdout:2006 write 0x906ea00 thru 0x907de0f (0xf410 bytes) 2024-09-17T23:30:00.446 INFO:teuthology.orchestra.run.smithi139.stdout:2008 read 0x48c4600 thru 0x48c4acb (0x4cc bytes) 2024-09-17T23:30:00.446 INFO:teuthology.orchestra.run.smithi139.stdout:2011 write 0x8069800 thru 0x807772f (0xdf30 bytes) 2024-09-17T23:30:00.451 INFO:teuthology.orchestra.run.smithi139.stdout:2012 trunc from 0xe3ac800 to 0xb2ac200 2024-09-17T23:30:00.471 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:00.471 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:00.473 INFO:teuthology.orchestra.run.smithi139.stdout:2013 read 0x1f32a00 thru 0x1f41c6c (0xf26d bytes) 2024-09-17T23:30:00.475 INFO:teuthology.orchestra.run.smithi139.stdout:2016 trunc from 0xb2ac200 to 0xc278c00 2024-09-17T23:30:00.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:00.496 INFO:teuthology.orchestra.run.smithi139.stdout:2017 trunc from 0xc278c00 to 0x6dc0000 2024-09-17T23:30:00.529 INFO:teuthology.orchestra.run.smithi139.stdout:2021 write 0xe0c2600 thru 0xe0c9a6f (0x7470 bytes) 2024-09-17T23:30:00.530 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:00.527+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:00.544 INFO:teuthology.orchestra.run.smithi139.stdout:2022 write 0xecf1200 thru 0xecfe16b (0xcf6c bytes) 2024-09-17T23:30:00.544 INFO:teuthology.orchestra.run.smithi139.stdout:2023 write 0x85e8400 thru 0x85ea527 (0x2128 bytes) 2024-09-17T23:30:00.545 INFO:teuthology.orchestra.run.smithi139.stdout:2025 write 0x36a00 thru 0x43e19 (0xd41a bytes) 2024-09-17T23:30:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:2026 write 0x7e9a000 thru 0x7ea8b8e (0xeb8f bytes) 2024-09-17T23:30:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:2027 write 0xb7d1200 thru 0xb7d5ff6 (0x4df7 bytes) 2024-09-17T23:30:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:2029 write 0xe418a00 thru 0xe4197ca (0xdcb bytes) 2024-09-17T23:30:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:2030 write 0xd682000 thru 0xd68ccba (0xacbb bytes) 2024-09-17T23:30:00.547 INFO:teuthology.orchestra.run.smithi139.stdout:2032 punch from 0xdee1800 to 0xdee7200, (0x5a00 bytes) 2024-09-17T23:30:00.554 INFO:teuthology.orchestra.run.smithi139.stdout:2034 trunc from 0xecfe200 to 0x947de00 2024-09-17T23:30:00.582 INFO:teuthology.orchestra.run.smithi139.stdout:2035 write 0x764a200 thru 0x76524b1 (0x82b2 bytes) 2024-09-17T23:30:00.583 INFO:teuthology.orchestra.run.smithi139.stdout:2038 punch from 0x158f600 to 0x1591000, (0x1a00 bytes) 2024-09-17T23:30:00.589 INFO:teuthology.orchestra.run.smithi139.stdout:2039 trunc from 0x947de00 to 0x49aee00 2024-09-17T23:30:00.625 INFO:teuthology.orchestra.run.smithi139.stdout:2042 write 0xdd32c00 thru 0xdd33961 (0xd62 bytes) 2024-09-17T23:30:00.631 INFO:teuthology.orchestra.run.smithi139.stdout:2043 read 0x895e600 thru 0x8960a81 (0x2482 bytes) 2024-09-17T23:30:00.631 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:00.628+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:00.635 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:00.635 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:00.641 INFO:teuthology.orchestra.run.smithi139.stdout:2044 trunc from 0xdd33a00 to 0x4e56400 2024-09-17T23:30:00.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:00.660 INFO:teuthology.orchestra.run.smithi139.stdout:2046 write 0x1d5200 thru 0x1de261 (0x9062 bytes) 2024-09-17T23:30:00.673 INFO:teuthology.orchestra.run.smithi139.stdout:2048 trunc from 0x4e56400 to 0xd643e00 2024-09-17T23:30:00.684 INFO:teuthology.orchestra.run.smithi139.stdout:2049 punch from 0x9640000 to 0x9648e00, (0x8e00 bytes) 2024-09-17T23:30:00.694 INFO:teuthology.orchestra.run.smithi139.stdout:2050 trunc from 0xd643e00 to 0xa101000 2024-09-17T23:30:00.704 INFO:teuthology.orchestra.run.smithi139.stdout:2053 punch from 0x62c400 to 0x633e00, (0x7a00 bytes) 2024-09-17T23:30:00.710 INFO:teuthology.orchestra.run.smithi139.stdout:2054 trunc from 0xa101000 to 0x65bd800 2024-09-17T23:30:00.723 INFO:teuthology.orchestra.run.smithi139.stdout:2057 read 0x4f8c000 thru 0x4f96d05 (0xad06 bytes) 2024-09-17T23:30:00.725 INFO:teuthology.orchestra.run.smithi139.stdout:2059 punch from 0x2197a00 to 0x2199e00, (0x2400 bytes) 2024-09-17T23:30:00.725 INFO:teuthology.orchestra.run.smithi139.stdout:2060 read 0x4e52a00 thru 0x4e5b561 (0x8b62 bytes) 2024-09-17T23:30:00.725 INFO:teuthology.orchestra.run.smithi139.stdout:2063 read 0x1600200 thru 0x1601fbc (0x1dbd bytes) 2024-09-17T23:30:00.726 INFO:teuthology.orchestra.run.smithi139.stdout:2065 write 0x2fbde00 thru 0x2fc9bf3 (0xbdf4 bytes) 2024-09-17T23:30:00.726 INFO:teuthology.orchestra.run.smithi139.stdout:2066 punch from 0xc0c600 to 0xc15a00, (0x9400 bytes) 2024-09-17T23:30:00.727 INFO:teuthology.orchestra.run.smithi139.stdout:2067 write 0x5d81000 thru 0x5d8641f (0x5420 bytes) 2024-09-17T23:30:00.727 INFO:teuthology.orchestra.run.smithi139.stdout:2068 write 0x22db200 thru 0x22e1ea8 (0x6ca9 bytes) 2024-09-17T23:30:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:2069 read 0x4210000 thru 0x421e112 (0xe113 bytes) 2024-09-17T23:30:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:2070 write 0x2a5dc00 thru 0x2a6da0f (0xfe10 bytes) 2024-09-17T23:30:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:2072 punch from 0x5bf9a00 to 0x5c00a00, (0x7000 bytes) 2024-09-17T23:30:00.732 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:00.729+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:00.734 INFO:teuthology.orchestra.run.smithi139.stdout:2076 trunc from 0x65bd800 to 0x9219a00 2024-09-17T23:30:00.741 INFO:teuthology.orchestra.run.smithi139.stdout:2077 read 0x79c2400 thru 0x79cc6d4 (0xa2d5 bytes) 2024-09-17T23:30:00.749 INFO:teuthology.orchestra.run.smithi139.stdout:2078 read 0x706400 thru 0x70a016 (0x3c17 bytes) 2024-09-17T23:30:00.750 INFO:teuthology.orchestra.run.smithi139.stdout:2080 read 0x808a000 thru 0x80916d7 (0x76d8 bytes) 2024-09-17T23:30:00.750 INFO:teuthology.orchestra.run.smithi139.stdout:2082 read 0x79ea800 thru 0x79f6af1 (0xc2f2 bytes) 2024-09-17T23:30:00.750 INFO:teuthology.orchestra.run.smithi139.stdout:2083 write 0x6bf2400 thru 0x6bf326c (0xe6d bytes) 2024-09-17T23:30:00.750 INFO:teuthology.orchestra.run.smithi139.stdout:2084 read 0x5b1fa00 thru 0x5b2cb1b (0xd11c bytes) 2024-09-17T23:30:00.750 INFO:teuthology.orchestra.run.smithi139.stdout:2086 write 0x3f23e00 thru 0x3f2c71d (0x891e bytes) 2024-09-17T23:30:00.751 INFO:teuthology.orchestra.run.smithi139.stdout:2088 write 0x2e71e00 thru 0x2e80d9b (0xef9c bytes) 2024-09-17T23:30:00.758 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:00.758 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:00.768 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:00.771 INFO:teuthology.orchestra.run.smithi139.stdout:2091 write 0xa74e200 thru 0xa75aaf7 (0xc8f8 bytes) 2024-09-17T23:30:00.771 INFO:teuthology.orchestra.run.smithi139.stdout:2092 punch from 0x5cfe00 to 0x5dc600, (0xc800 bytes) 2024-09-17T23:30:00.778 INFO:teuthology.orchestra.run.smithi139.stdout:2094 trunc from 0xa75ac00 to 0x4681400 2024-09-17T23:30:00.816 INFO:teuthology.orchestra.run.smithi139.stdout:2097 write 0x7056800 thru 0x705ee4f (0x8650 bytes) 2024-09-17T23:30:00.821 INFO:teuthology.orchestra.run.smithi139.stdout:2099 read 0x808000 thru 0x80b49c (0x349d bytes) 2024-09-17T23:30:00.822 INFO:teuthology.orchestra.run.smithi139.stdout:2100 write 0x3077600 thru 0x3086f98 (0xf999 bytes) 2024-09-17T23:30:00.822 INFO:teuthology.orchestra.run.smithi139.stdout:2104 read 0x32f9a00 thru 0x3302cfa (0x92fb bytes) 2024-09-17T23:30:00.823 INFO:teuthology.orchestra.run.smithi139.stdout:2105 clone 19 order 18 su 16384 sc 10 2024-09-17T23:30:00.833 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:00.831+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:00.849 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:30:00.883 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:00.884 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:00.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:00.935 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:00.932+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:01.036 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:01.033+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:01.041 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.042 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:01.136 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:01.134+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:01.161 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.161 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:01.230 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T23:30:01.237 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:01.235+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:01.282 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.283 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:01.325 INFO:teuthology.orchestra.run.smithi139.stdout:2106 write 0x74c1e00 thru 0x74c8b7c (0x6d7d bytes) 2024-09-17T23:30:01.338 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:01.335+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:01.349 INFO:teuthology.orchestra.run.smithi139.stdout:2107 write 0x81e9000 thru 0x81f0694 (0x7695 bytes) 2024-09-17T23:30:01.381 INFO:teuthology.orchestra.run.smithi139.stdout:2109 write 0xdc09800 thru 0xdc18c56 (0xf457 bytes) 2024-09-17T23:30:01.396 INFO:teuthology.orchestra.run.smithi139.stdout:2111 trunc from 0xdc18e00 to 0x11a9000 2024-09-17T23:30:01.416 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.416 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:01.439 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:01.437+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:01.456 INFO:teuthology.orchestra.run.smithi139.stdout:2112 write 0x555b200 thru 0x555f9fd (0x47fe bytes) 2024-09-17T23:30:01.456 INFO:teuthology.orchestra.run.smithi139.stdout:2117 read 0xbff600 thru 0xc02c21 (0x3622 bytes) 2024-09-17T23:30:01.460 INFO:teuthology.orchestra.run.smithi139.stdout:2118 read 0x1271c00 thru 0x1273a7a (0x1e7b bytes) 2024-09-17T23:30:01.461 INFO:teuthology.orchestra.run.smithi139.stdout:2119 read 0x4307e00 thru 0x4316360 (0xe561 bytes) 2024-09-17T23:30:01.461 INFO:teuthology.orchestra.run.smithi139.stdout:2120 read 0x4126a00 thru 0x4133821 (0xce22 bytes) 2024-09-17T23:30:01.461 INFO:teuthology.orchestra.run.smithi139.stdout:2121 read 0x5002200 thru 0x500c521 (0xa322 bytes) 2024-09-17T23:30:01.461 INFO:teuthology.orchestra.run.smithi139.stdout:2122 punch from 0x49a2600 to 0x49a3600, (0x1000 bytes) 2024-09-17T23:30:01.471 INFO:teuthology.orchestra.run.smithi139.stdout:2123 trunc from 0x555fa00 to 0xdfa3000 2024-09-17T23:30:01.482 INFO:teuthology.orchestra.run.smithi139.stdout:2126 read 0xc8e5c00 thru 0xc8f5261 (0xf662 bytes) 2024-09-17T23:30:01.496 INFO:teuthology.orchestra.run.smithi139.stdout:2128 trunc from 0xdfa3000 to 0x2a67000 2024-09-17T23:30:01.514 INFO:teuthology.orchestra.run.smithi139.stdout:2130 punch from 0x263b000 to 0x2646000, (0xb000 bytes) 2024-09-17T23:30:01.541 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:01.538+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:01.544 INFO:teuthology.orchestra.run.smithi139.stdout:2131 write 0xe7ab000 thru 0xe7b0ae3 (0x5ae4 bytes) 2024-09-17T23:30:01.552 INFO:teuthology.orchestra.run.smithi139.stdout:2133 read 0x7c71a00 thru 0x7c72cd6 (0x12d7 bytes) 2024-09-17T23:30:01.552 INFO:teuthology.orchestra.run.smithi139.stdout:2134 write 0x40f5e00 thru 0x40faf71 (0x5172 bytes) 2024-09-17T23:30:01.553 INFO:teuthology.orchestra.run.smithi139.stdout:2135 write 0x1eec400 thru 0x1efb5e2 (0xf1e3 bytes) 2024-09-17T23:30:01.553 INFO:teuthology.orchestra.run.smithi139.stdout:2137 read 0x9bab000 thru 0x9bb901f (0xe020 bytes) 2024-09-17T23:30:01.553 INFO:teuthology.orchestra.run.smithi139.stdout:2139 read 0xe35ca00 thru 0xe36493b (0x7f3c bytes) 2024-09-17T23:30:01.553 INFO:teuthology.orchestra.run.smithi139.stdout:2140 trunc from 0xe7b0c00 to 0xdbfc800 2024-09-17T23:30:01.590 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.591 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:01.642 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:01.639+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:01.742 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:01.740+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:01.747 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.747 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:01.843 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:01.840+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:01.867 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:01.867 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:01.875 INFO:teuthology.orchestra.run.smithi139.stdout:2141 write 0x765ec00 thru 0x766c4e2 (0xd8e3 bytes) 2024-09-17T23:30:01.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:01.883 INFO:teuthology.orchestra.run.smithi139.stdout:2145 punch from 0xbfdb800 to 0xbfe4400, (0x8c00 bytes) 2024-09-17T23:30:01.883 INFO:teuthology.orchestra.run.smithi139.stdout:2148 read 0xc1a4a00 thru 0xc1b008f (0xb690 bytes) 2024-09-17T23:30:01.883 INFO:teuthology.orchestra.run.smithi139.stdout:2151 write 0x2107000 thru 0x2110763 (0x9764 bytes) 2024-09-17T23:30:01.884 INFO:teuthology.orchestra.run.smithi139.stdout:2154 read 0x40cba00 thru 0x40d1bbc (0x61bd bytes) 2024-09-17T23:30:01.885 INFO:teuthology.orchestra.run.smithi139.stdout:2156 read 0x89b4e00 thru 0x89bc203 (0x7404 bytes) 2024-09-17T23:30:01.885 INFO:teuthology.orchestra.run.smithi139.stdout:2157 read 0x3d0b800 thru 0x3d0cdae (0x15af bytes) 2024-09-17T23:30:01.911 INFO:teuthology.orchestra.run.smithi139.stdout:2160 write 0xe63ec00 thru 0xe63f319 (0x71a bytes) 2024-09-17T23:30:01.913 INFO:teuthology.orchestra.run.smithi139.stdout:2163 write 0x7b1f000 thru 0x7b27408 (0x8409 bytes) 2024-09-17T23:30:01.914 INFO:teuthology.orchestra.run.smithi139.stdout:2164 read 0x51d5e00 thru 0x51d8368 (0x2569 bytes) 2024-09-17T23:30:01.914 INFO:teuthology.orchestra.run.smithi139.stdout:2165 read 0xa3b0600 thru 0xa3b23c1 (0x1dc2 bytes) 2024-09-17T23:30:01.914 INFO:teuthology.orchestra.run.smithi139.stdout:2166 write 0x4b3d600 thru 0x4b45923 (0x8324 bytes) 2024-09-17T23:30:01.917 INFO:teuthology.orchestra.run.smithi139.stdout:2167 trunc from 0xe63f400 to 0xc04e000 2024-09-17T23:30:01.944 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:01.942+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:01.951 INFO:teuthology.orchestra.run.smithi139.stdout:2169 write 0x8081600 thru 0x8088672 (0x7073 bytes) 2024-09-17T23:30:01.951 INFO:teuthology.orchestra.run.smithi139.stdout:2170 write 0x151e400 thru 0x15243d6 (0x5fd7 bytes) 2024-09-17T23:30:01.951 INFO:teuthology.orchestra.run.smithi139.stdout:2172 write 0x55e2600 thru 0x55f131b (0xed1c bytes) 2024-09-17T23:30:01.951 INFO:teuthology.orchestra.run.smithi139.stdout:2174 punch from 0x885ee00 to 0x886c200, (0xd400 bytes) 2024-09-17T23:30:01.958 INFO:teuthology.orchestra.run.smithi139.stdout:2175 trunc from 0xc04e000 to 0x6c6ec00 2024-09-17T23:30:01.995 INFO:teuthology.orchestra.run.smithi139.stdout:2180 trunc from 0x6c6ec00 to 0x18b8e00 2024-09-17T23:30:02.023 INFO:teuthology.orchestra.run.smithi139.stdout:2181 trunc from 0x18b8e00 to 0x2598400 2024-09-17T23:30:02.033 INFO:teuthology.orchestra.run.smithi139.stdout:2182 punch from 0x9c3200 to 0x9d3000, (0xfe00 bytes) 2024-09-17T23:30:02.037 INFO:teuthology.orchestra.run.smithi139.stdout:2184 read 0x1430a00 thru 0x143b4f0 (0xaaf1 bytes) 2024-09-17T23:30:02.039 INFO:teuthology.orchestra.run.smithi139.stdout:2186 trunc from 0x2598400 to 0x2f8f000 2024-09-17T23:30:02.044 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:02.043+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:02.054 INFO:teuthology.orchestra.run.smithi139.stdout:2188 read 0x8a2200 thru 0x8a4c6c (0x2a6d bytes) 2024-09-17T23:30:02.057 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:02.058 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:02.059 INFO:teuthology.orchestra.run.smithi139.stdout:2189 read 0x6a3200 thru 0x6a359d (0x39e bytes) 2024-09-17T23:30:02.059 INFO:teuthology.orchestra.run.smithi139.stdout:2190 read 0x2e01a00 thru 0x2e0d887 (0xbe88 bytes) 2024-09-17T23:30:02.059 INFO:teuthology.orchestra.run.smithi139.stdout:2191 punch from 0x298f400 to 0x2994c00, (0x5800 bytes) 2024-09-17T23:30:02.059 INFO:teuthology.orchestra.run.smithi139.stdout:2192 read 0x1e07400 thru 0x1e074aa (0xab bytes) 2024-09-17T23:30:02.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:02.074 INFO:teuthology.orchestra.run.smithi139.stdout:2195 write 0x4290800 thru 0x4295678 (0x4e79 bytes) 2024-09-17T23:30:02.080 INFO:teuthology.orchestra.run.smithi139.stdout:2197 read 0x219fe00 thru 0x21a7fef (0x81f0 bytes) 2024-09-17T23:30:02.080 INFO:teuthology.orchestra.run.smithi139.stdout:2201 read 0x219b000 thru 0x21a9df2 (0xedf3 bytes) 2024-09-17T23:30:02.080 INFO:teuthology.orchestra.run.smithi139.stdout:2202 punch from 0x928600 to 0x92da00, (0x5400 bytes) 2024-09-17T23:30:02.080 INFO:teuthology.orchestra.run.smithi139.stdout:2203 write 0x2502400 thru 0x250edbf (0xc9c0 bytes) 2024-09-17T23:30:02.080 INFO:teuthology.orchestra.run.smithi139.stdout:2205 punch from 0x4b3400 to 0x4c1a00, (0xe600 bytes) 2024-09-17T23:30:02.082 INFO:teuthology.orchestra.run.smithi139.stdout:2209 trunc from 0x4295800 to 0x2fe7a00 2024-09-17T23:30:02.122 INFO:teuthology.orchestra.run.smithi139.stdout:2210 write 0x46f6400 thru 0x4704f9e (0xeb9f bytes) 2024-09-17T23:30:02.128 INFO:teuthology.orchestra.run.smithi139.stdout:2211 write 0x21a2a00 thru 0x21aa5d4 (0x7bd5 bytes) 2024-09-17T23:30:02.132 INFO:teuthology.orchestra.run.smithi139.stdout:2217 trunc from 0x4705000 to 0x5dc1e00 2024-09-17T23:30:02.146 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:02.144+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:02.152 INFO:teuthology.orchestra.run.smithi139.stdout:2219 read 0x28a8a00 thru 0x28b3db6 (0xb3b7 bytes) 2024-09-17T23:30:02.175 INFO:teuthology.orchestra.run.smithi139.stdout:2220 write 0xb345400 thru 0xb348946 (0x3547 bytes) 2024-09-17T23:30:02.180 INFO:teuthology.orchestra.run.smithi139.stdout:2222 read 0x7333000 thru 0x733a6ad (0x76ae bytes) 2024-09-17T23:30:02.184 INFO:teuthology.orchestra.run.smithi139.stdout:2226 trunc from 0xb348a00 to 0x7f94e00 2024-09-17T23:30:02.192 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:02.192 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:02.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:02.204 INFO:teuthology.orchestra.run.smithi139.stdout:2229 read 0x4d25c00 thru 0x4d2e49b (0x889c bytes) 2024-09-17T23:30:02.226 INFO:teuthology.orchestra.run.smithi139.stdout:2230 write 0xd910c00 thru 0xd91bd55 (0xb156 bytes) 2024-09-17T23:30:02.231 INFO:teuthology.orchestra.run.smithi139.stdout:2233 punch from 0x50f2e00 to 0x50f3e00, (0x1000 bytes) 2024-09-17T23:30:02.232 INFO:teuthology.orchestra.run.smithi139.stdout:2235 write 0x6724a00 thru 0x672efe5 (0xa5e6 bytes) 2024-09-17T23:30:02.232 INFO:teuthology.orchestra.run.smithi139.stdout:2238 write 0x30b3e00 thru 0x30bfd5e (0xbf5f bytes) 2024-09-17T23:30:02.232 INFO:teuthology.orchestra.run.smithi139.stdout:2240 read 0x1b47200 thru 0x1b47a1e (0x81f bytes) 2024-09-17T23:30:02.237 INFO:teuthology.orchestra.run.smithi139.stdout:2241 trunc from 0xd91be00 to 0x9351e00 2024-09-17T23:30:02.246 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:02.244+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:02.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:02.264 INFO:teuthology.orchestra.run.smithi139.stdout:2244 punch from 0x6597800 to 0x659e200, (0x6a00 bytes) 2024-09-17T23:30:02.269 INFO:teuthology.orchestra.run.smithi139.stdout:2246 punch from 0xb4e600 to 0xb57200, (0x8c00 bytes) 2024-09-17T23:30:02.270 INFO:teuthology.orchestra.run.smithi139.stdout:2247 write 0x3cc7e00 thru 0x3cd63e9 (0xe5ea bytes) 2024-09-17T23:30:02.297 INFO:teuthology.orchestra.run.smithi139.stdout:2249 write 0xeba7200 thru 0xebb3b38 (0xc939 bytes) 2024-09-17T23:30:02.313 INFO:teuthology.orchestra.run.smithi139.stdout:2250 trunc from 0xebb3c00 to 0x189a600 2024-09-17T23:30:02.332 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:02.333 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:02.347 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:02.347 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:02.344+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:02.375 INFO:teuthology.orchestra.run.smithi139.stdout:2252 trunc from 0x189a600 to 0x7ec8a00 2024-09-17T23:30:02.405 INFO:teuthology.orchestra.run.smithi139.stdout:2254 write 0xcc41a00 thru 0xcc4c3ed (0xa9ee bytes) 2024-09-17T23:30:02.410 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:02.410 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:02.411 INFO:teuthology.orchestra.run.smithi139.stdout:2255 write 0x8762000 thru 0x8765ecf (0x3ed0 bytes) 2024-09-17T23:30:02.412 INFO:teuthology.orchestra.run.smithi139.stdout:2256 punch from 0x8ee1c00 to 0x8ee6400, (0x4800 bytes) 2024-09-17T23:30:02.412 INFO:teuthology.orchestra.run.smithi139.stdout:2258 read 0x3aaba00 thru 0x3ab3ef0 (0x84f1 bytes) 2024-09-17T23:30:02.412 INFO:teuthology.orchestra.run.smithi139.stdout:2259 punch from 0x9bfac00 to 0x9c0a000, (0xf400 bytes) 2024-09-17T23:30:02.412 INFO:teuthology.orchestra.run.smithi139.stdout:2260 write 0x893ce00 thru 0x8943183 (0x6384 bytes) 2024-09-17T23:30:02.426 INFO:teuthology.orchestra.run.smithi139.stdout:2261 trunc from 0xcc4c400 to 0xbc8800 2024-09-17T23:30:02.433 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:02.434 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:02.449 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:02.445+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:02.461 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:02.462 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:02.483 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:02.483 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:02.500 INFO:teuthology.orchestra.run.smithi139.stdout:2263 write 0x27b2400 thru 0x27b9258 (0x6e59 bytes) 2024-09-17T23:30:02.504 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:02.525 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:02.525 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:02.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14760 3.11 3:8ddf073a:::rbd_data.1152c1142907.0000000000000029:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.487099682, 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010811016999999999, 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14760 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034623+0000", 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034627+0000", 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4680000000000003e-06 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034635+0000", 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034641+0000", 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035590+0000", 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094941100000000005 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036421+0000", 2024-09-17T23:30:02.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083071299999999996 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036612+0000", 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191342 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041698+0000", 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050861980000000001 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045354+0000", 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036561440000000001 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045394+0000", 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9604999999999998e-05 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045433+0000", 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8945000000000001e-05 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.486065747000001, 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:02.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:02.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:02.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:02.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:02.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:02.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:02.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:02.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.485892806999999, 2024-09-17T23:30:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:02.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:02.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:02.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.485852937000001, 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:02.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:02.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:02.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14758 3.1 e41/33 3:853c0e15:::rbd_data.1152c1142907.0000000000000028:head v 41'194, mlcod=39'189)", 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.484488567, 2024-09-17T23:30:02.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010716488999999999, 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037233+0000", 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037234+0000", 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5000000000000002e-07 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037239+0000", 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.784e-06 2024-09-17T23:30:02.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037249+0000", 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8889999999999997e-06 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037257+0000", 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.379e-06 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038255+0000", 2024-09-17T23:30:02.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099873299999999991 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038288+0000", 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292000000000002e-05 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047928+0000", 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096399710000000007 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047950+0000", 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1790999999999999e-05 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21686 3.11 3:8d00fb61:::rbd_data.1152c1142907.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.194541934, 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010211289, 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21686 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327180+0000", 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327182+0000", 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.199e-06 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327654+0000", 2024-09-17T23:30:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000472094 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327668+0000", 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3867e-05 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327676+0000", 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9289999999999998e-06 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327897+0000", 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022101 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.327979+0000", 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1921000000000004e-05 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.328181+0000", 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020200799999999999 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337270+0000", 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090895820000000006 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337286+0000", 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6112000000000001e-05 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337344+0000", 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7989999999999999e-05 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337391+0000", 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6576999999999998e-05 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21695 3.11 3:883560f3:::rbd_data.1152c1142907.000000000000001f:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.192529450999999, 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010154599, 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21695 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329192+0000", 2024-09-17T23:30:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329193+0000", 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4499999999999995e-07 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329198+0000", 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329213+0000", 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4606999999999999e-05 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.329220+0000", 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.328e-06 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331593+0000", 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002373287 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331670+0000", 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6156999999999996e-05 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.331868+0000", 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019856600000000001 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.337649+0000", 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057810889999999997 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339251+0000", 2024-09-17T23:30:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016017309999999999 2024-09-17T23:30:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339305+0000", 2024-09-17T23:30:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3594000000000002e-05 2024-09-17T23:30:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339347+0000", 2024-09-17T23:30:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2209999999999997e-05 2024-09-17T23:30:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.19134871, 2024-09-17T23:30:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.943647161000001, 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9337148949999996, 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9204878460000003, 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:288 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141710+0000", 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3800113199999999, 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010374023, 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 288 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141710+0000", 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141710+0000", 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141711+0000", 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-07 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141714+0000", 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.712e-06 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141721+0000", 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5310000000000004e-06 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141727+0000", 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8200000000000002e-06 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142096+0000", 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036878300000000002 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142099+0000", 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7129999999999999e-06 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.151832+0000", 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097322789999999999 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.151923+0000", 2024-09-17T23:30:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1386999999999998e-05 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152084+0000", 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000161268 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:289 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3799816539999998, 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010772220000000001, 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 289 2024-09-17T23:30:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141743+0000", 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141750+0000", 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-06 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141755+0000", 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142121+0000", 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036654199999999997 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142125+0000", 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.968e-06 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152136+0000", 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010011124 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152323+0000", 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018617699999999999 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152512+0000", 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001896 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.37995869, 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3799372999999999, 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3799201710000002, 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3799028890000002, 2024-09-17T23:30:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3798858599999999, 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3798529859999999, 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.3792652799999998, 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:02.665 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:02.666 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:02.666 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:02.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:02.674 INFO:teuthology.orchestra.run.smithi139.stdout:2264 write 0x80e2c00 thru 0x80edcde (0xb0df bytes) 2024-09-17T23:30:02.674 INFO:teuthology.orchestra.run.smithi139.stdout:2265 read 0x4c43e00 thru 0x4c4e194 (0xa395 bytes) 2024-09-17T23:30:02.674 INFO:teuthology.orchestra.run.smithi139.stdout:2266 trunc from 0x80ede00 to 0xa2c0400 2024-09-17T23:30:02.674 INFO:teuthology.orchestra.run.smithi139.stdout:2269 read 0x3c90600 thru 0x3c9b64c (0xb04d bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2270 punch from 0x29ca600 to 0x29d0e00, (0x6800 bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2272 read 0x7632a00 thru 0x76342e1 (0x18e2 bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2273 write 0x8941a00 thru 0x894628d (0x488e bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2274 trunc from 0xa2c0400 to 0xb965a00 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2275 write 0x65efa00 thru 0x65fec8b (0xf28c bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2278 write 0x1c90200 thru 0x1c9197f (0x1780 bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2279 write 0xc405200 thru 0xc4125f9 (0xd3fa bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2282 punch from 0x6211600 to 0x621c400, (0xae00 bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2283 trunc from 0xc412600 to 0xb2a6c00 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2284 punch from 0xab2c600 to 0xab3c000, (0xfa00 bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2285 write 0x5193600 thru 0x519726c (0x3c6d bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2286 read 0x71a4200 thru 0x71b01c5 (0xbfc6 bytes) 2024-09-17T23:30:02.675 INFO:teuthology.orchestra.run.smithi139.stdout:2289 read 0x1416600 thru 0x1423e81 (0xd882 bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2291 read 0x449fa00 thru 0x44aa804 (0xae05 bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2293 read 0x99c4e00 thru 0x99c5bbb (0xdbc bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2294 write 0x4c01000 thru 0x4c0e681 (0xd682 bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2296 write 0x18afe00 thru 0x18b64ea (0x66eb bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2300 punch from 0x6492e00 to 0x64a1c00, (0xee00 bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2301 punch from 0x475fc00 to 0x4768000, (0x8400 bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2304 read 0x4545e00 thru 0x4546953 (0xb54 bytes) 2024-09-17T23:30:02.676 INFO:teuthology.orchestra.run.smithi139.stdout:2306 read 0x9b64000 thru 0x9b679a1 (0x39a2 bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2308 read 0x913a800 thru 0x914249e (0x7c9f bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2309 read 0x72cc400 thru 0x72cfdd7 (0x39d8 bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2310 punch from 0x84cc800 to 0x84cca00, (0x200 bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2316 punch from 0x5947600 to 0x594aa00, (0x3400 bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2317 read 0x991a400 thru 0x9921396 (0x6f97 bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2318 read 0x83eec00 thru 0x83fddfc (0xf1fd bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2322 read 0x2fc0a00 thru 0x2fc8daa (0x83ab bytes) 2024-09-17T23:30:02.677 INFO:teuthology.orchestra.run.smithi139.stdout:2324 trunc from 0xb2a6c00 to 0x131e00 2024-09-17T23:30:02.678 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:02.668+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:02.695 INFO:teuthology.orchestra.run.smithi139.stdout:2328 punch from 0x89000 to 0x8f400, (0x6400 bytes) 2024-09-17T23:30:02.714 INFO:teuthology.orchestra.run.smithi139.stdout:2330 write 0x1529800 thru 0x153120f (0x7a10 bytes) 2024-09-17T23:30:02.749 INFO:teuthology.orchestra.run.smithi139.stdout:2333 write 0xc7c1e00 thru 0xc7c7d39 (0x5f3a bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2335 read 0x7108a00 thru 0x711280a (0x9e0b bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2336 write 0xa33fc00 thru 0xa345139 (0x553a bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2337 read 0x6471c00 thru 0x647c119 (0xa51a bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2338 punch from 0x5a74800 to 0x5a7d600, (0x8e00 bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2339 read 0x62e5400 thru 0x62ed935 (0x8536 bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2341 read 0x6c4c600 thru 0x6c4f857 (0x3258 bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2342 write 0x7b40800 thru 0x7b4b248 (0xaa49 bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2343 write 0xfa2400 thru 0xfa93ea (0x6feb bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2344 read 0xafe3000 thru 0xafe9279 (0x627a bytes) 2024-09-17T23:30:02.759 INFO:teuthology.orchestra.run.smithi139.stdout:2345 write 0x9703200 thru 0x9708b3d (0x593e bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2346 read 0x5fec200 thru 0x5fedb34 (0x1935 bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2352 read 0xbcec800 thru 0xbcfbba3 (0xf3a4 bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2353 read 0x72c8e00 thru 0x72d3abf (0xacc0 bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2354 punch from 0x12e4200 to 0x12f3400, (0xf200 bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2355 punch from 0xda6600 to 0xdac000, (0x5a00 bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2356 write 0x1679e00 thru 0x1687f9d (0xe19e bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2357 read 0x7639600 thru 0x7644ce7 (0xb6e8 bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2358 read 0x7825800 thru 0x782f0eb (0x98ec bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2359 write 0x2e93400 thru 0x2e9e70e (0xb30f bytes) 2024-09-17T23:30:02.760 INFO:teuthology.orchestra.run.smithi139.stdout:2361 write 0x109000 thru 0x118b2e (0xfb2f bytes) 2024-09-17T23:30:02.762 INFO:teuthology.orchestra.run.smithi139.stdout:2363 read 0x9ac6400 thru 0x9acf5f1 (0x91f2 bytes) 2024-09-17T23:30:02.762 INFO:teuthology.orchestra.run.smithi139.stdout:2364 write 0x4186400 thru 0x4192588 (0xc189 bytes) 2024-09-17T23:30:02.762 INFO:teuthology.orchestra.run.smithi139.stdout:2366 read 0x1e14200 thru 0x1e18062 (0x3e63 bytes) 2024-09-17T23:30:02.762 INFO:teuthology.orchestra.run.smithi139.stdout:2368 read 0x2779c00 thru 0x2783921 (0x9d22 bytes) 2024-09-17T23:30:02.762 INFO:teuthology.orchestra.run.smithi139.stdout:2369 punch from 0x1be2000 to 0x1bea600, (0x8600 bytes) 2024-09-17T23:30:02.762 INFO:teuthology.orchestra.run.smithi139.stdout:2370 read 0x1b73200 thru 0x1b76bd1 (0x39d2 bytes) 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:2371 read 0x629aa00 thru 0x62a1218 (0x6819 bytes) 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:2372 write 0x3213400 thru 0x3217127 (0x3d28 bytes) 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:2373 punch from 0x2ca9e00 to 0x2cb1800, (0x7a00 bytes) 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:2374 punch from 0x1db4000 to 0x1db7c00, (0x3c00 bytes) 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:2378 read 0xd38400 thru 0xd3edfc (0x69fd bytes) 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-17T23:30:02.763 INFO:teuthology.orchestra.run.smithi139.stdout:2385 trunc from 0xc7c7e00 to 0xd8b7600 2024-09-17T23:30:02.770 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:02.768+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5079.0:4431 3.1e 3:7bc308f0:::rbd_data.11526e6e10a0.0000000000000108:head [set-alloc-hint object_size 262144 write_size 262144,write 49152~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:02.793199+0000", 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0013642680000000001, 2024-09-17T23:30:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013829129999999999, 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for sub ops", 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5079", 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3723618240", 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4431 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793199+0000", 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793199+0000", 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793200+0000", 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.592e-06 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793215+0000", 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5073e-05 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793234+0000", 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8270999999999998e-05 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793241+0000", 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9140000000000008e-06 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793299+0000", 2024-09-17T23:30:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7624000000000003e-05 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793438+0000", 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013875500000000001 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793581+0000", 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000143141 2024-09-17T23:30:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.794454+0000", 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00087275600000000003 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5079.0:4432 3.1b 3:da0be834:::rbd_data.11526e6e10a0.0000000000000109:head [set-alloc-hint object_size 262144 write_size 262144,write 49152~16384 in=16384b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-17T23:30:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:02.793332+0000", 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0012309090000000001, 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001356335, 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for sub ops", 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5079", 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3723618240", 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4432 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793332+0000", 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793332+0000", 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793333+0000", 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.122e-06 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793349+0000", 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5736999999999999e-05 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793361+0000", 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2215000000000001e-05 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793369+0000", 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5839999999999997e-06 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793427+0000", 2024-09-17T23:30:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8183000000000002e-05 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793561+0000", 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000133786 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:02.793718+0000", 2024-09-17T23:30:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000157594 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 2 2024-09-17T23:30:02.808 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:02.809 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:02.812 INFO:teuthology.orchestra.run.smithi139.stdout:2386 read 0x5cf2400 thru 0x5cfb75c (0x935d bytes) 2024-09-17T23:30:02.827 INFO:teuthology.orchestra.run.smithi139.stdout:2388 trunc from 0xd8b7600 to 0x7844a00 2024-09-17T23:30:02.848 INFO:teuthology.orchestra.run.smithi139.stdout:2389 flatten 2024-09-17T23:30:02.859 INFO:teuthology.orchestra.run.smithi139.stdout:2391 read 0x7713600 thru 0x771c1e3 (0x8be4 bytes) 2024-09-17T23:30:02.859 INFO:teuthology.orchestra.run.smithi139.stdout:2392 read 0x35a7e00 thru 0x35b72c5 (0xf4c6 bytes) 2024-09-17T23:30:02.859 INFO:teuthology.orchestra.run.smithi139.stdout:2393 read 0x2e48200 thru 0x2e4d3fe (0x51ff bytes) 2024-09-17T23:30:02.864 INFO:teuthology.orchestra.run.smithi139.stdout:2395 trunc from 0x7844a00 to 0xb0f4000 2024-09-17T23:30:02.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:30:02.871 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:02.869+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:02.875 INFO:teuthology.orchestra.run.smithi139.stdout:2396 write 0x5ff9000 thru 0x6006fcb (0xdfcc bytes) 2024-09-17T23:30:02.881 INFO:teuthology.orchestra.run.smithi139.stdout:2397 write 0x4dbae00 thru 0x4dbcc9e (0x1e9f bytes) 2024-09-17T23:30:02.881 INFO:teuthology.orchestra.run.smithi139.stdout:2398 write 0x81c0400 thru 0x81ca40e (0xa00f bytes) 2024-09-17T23:30:02.882 INFO:teuthology.orchestra.run.smithi139.stdout:2400 read 0x99dc00 thru 0x9a5b72 (0x7f73 bytes) 2024-09-17T23:30:02.882 INFO:teuthology.orchestra.run.smithi139.stdout:2402 punch from 0x50da200 to 0x50e5600, (0xb400 bytes) 2024-09-17T23:30:02.882 INFO:teuthology.orchestra.run.smithi139.stdout:2405 write 0xd08800 thru 0xd0bfe8 (0x37e9 bytes) 2024-09-17T23:30:02.883 INFO:teuthology.orchestra.run.smithi139.stdout:2410 write 0x86d0000 thru 0x86d4376 (0x4377 bytes) 2024-09-17T23:30:02.883 INFO:teuthology.orchestra.run.smithi139.stdout:2411 punch from 0x3223400 to 0x3224e00, (0x1a00 bytes) 2024-09-17T23:30:02.883 INFO:teuthology.orchestra.run.smithi139.stdout:2413 read 0x5f5e400 thru 0x5f60227 (0x1e28 bytes) 2024-09-17T23:30:02.883 INFO:teuthology.orchestra.run.smithi139.stdout:2415 read 0xa29400 thru 0xa37d92 (0xe993 bytes) 2024-09-17T23:30:02.883 INFO:teuthology.orchestra.run.smithi139.stdout:2417 read 0x3e47000 thru 0x3e525b0 (0xb5b1 bytes) 2024-09-17T23:30:02.883 INFO:teuthology.orchestra.run.smithi139.stdout:2418 read 0x7656800 thru 0x765b618 (0x4e19 bytes) 2024-09-17T23:30:02.896 INFO:teuthology.orchestra.run.smithi139.stdout:2419 trunc from 0xb0f4000 to 0x1b0800 2024-09-17T23:30:02.933 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:02.934 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:02.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:02.972 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:02.969+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:02.985 INFO:teuthology.orchestra.run.smithi139.stdout:2420 write 0x52b7000 thru 0x52c2a6f (0xba70 bytes) 2024-09-17T23:30:02.986 INFO:teuthology.orchestra.run.smithi139.stdout:2421 read 0x3140a00 thru 0x314f977 (0xef78 bytes) 2024-09-17T23:30:03.007 INFO:teuthology.orchestra.run.smithi139.stdout:2422 write 0x8100c00 thru 0x810b195 (0xa596 bytes) 2024-09-17T23:30:03.008 INFO:teuthology.orchestra.run.smithi139.stdout:2423 read 0x5d44a00 thru 0x5d4d85d (0x8e5e bytes) 2024-09-17T23:30:03.013 INFO:teuthology.orchestra.run.smithi139.stdout:2424 trunc from 0x810b200 to 0xc2cb200 2024-09-17T23:30:03.041 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:03.041 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:03.041 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T23:30:03.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:30:03.042 INFO:teuthology.orchestra.run.smithi139.stdout:2426 trunc from 0xc2cb200 to 0x122f800 2024-09-17T23:30:03.066 INFO:teuthology.orchestra.run.smithi139.stdout:2427 read 0xd58200 thru 0xd5bee5 (0x3ce6 bytes) 2024-09-17T23:30:03.073 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:03.071+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:03.080 INFO:teuthology.orchestra.run.smithi139.stdout:2430 write 0x2c48400 thru 0x2c4d013 (0x4c14 bytes) 2024-09-17T23:30:03.081 INFO:teuthology.orchestra.run.smithi139.stdout:2431 punch from 0x141a000 to 0x141ce00, (0x2e00 bytes) 2024-09-17T23:30:03.093 INFO:teuthology.orchestra.run.smithi139.stdout:2433 trunc from 0x2c4d200 to 0xc9ddc00 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.220889036999999, 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205263907000003, 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205226250999999, 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205208820000003, 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205173889000001, 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205142469000002, 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205125842999998, 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205104149, 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.205092430999997, 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.640890087999999, 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:30:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.640867622000002, 2024-09-17T23:30:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:30:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3262 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.633422842999998, 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045783590999999998, 2024-09-17T23:30:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3262 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463817+0000", 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00160411 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463821+0000", 2024-09-17T23:30:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6670000000000004e-06 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463838+0000", 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6898999999999999e-05 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463848+0000", 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507787+0000", 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043939260000000001 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507895+0000", 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010859899999999999 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507996+0000", 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100587 2024-09-17T23:30:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.623529338000001, 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:30:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.621621443999999, 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:30:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.619636373999999, 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:30:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:30:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.619626779000001, 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:30:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12611 3.18 3:1c8edc2b:::rbd_data.115262cdab6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5863160589999996, 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.070059079999999996, 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12611 2024-09-17T23:30:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509321+0000", 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-06 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553867+0000", 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044545920000000003 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553893+0000", 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6094000000000001e-05 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553906+0000", 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3079e-05 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553972+0000", 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6630000000000004e-05 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.554090+0000", 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117364 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.555383+0000", 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001293188 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.577545+0000", 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.022161943 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579298+0000", 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017536349999999999 2024-09-17T23:30:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579346+0000", 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7858000000000002e-05 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579378+0000", 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2011000000000002e-05 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12614 3.2 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.5416952770000005, 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.100611516, 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12614 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553942+0000", 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.477e-06 2024-09-17T23:30:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599428+0000", 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045486208 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599454+0000", 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6551999999999999e-05 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599467+0000", 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2339e-05 2024-09-17T23:30:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599548+0000", 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599660+0000", 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011209699999999999 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.601086+0000", 2024-09-17T23:30:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014260550000000001 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.612776+0000", 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011690010000000001 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654410+0000", 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.041634517000000003 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654486+0000", 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6260999999999994e-05 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654552+0000", 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5086999999999995e-05 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4968.0:12613 3.9 e48/19 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head v 48'120, mlcod=45'117)", 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.4974371830000006, 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049712148999999997, 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598199+0000", 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-07 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637181+0000", 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.038982528000000002 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637209+0000", 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7317e-05 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637219+0000", 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0372e-05 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637307+0000", 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7867000000000004e-05 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637345+0000", 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7735999999999999e-05 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647845+0000", 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010500012 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647910+0000", 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5662999999999998e-05 2024-09-17T23:30:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12676 3.1b 3:d8a99671:::rbd_data.115262cdab6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.4911925690000007, 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.074765842999999998, 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12676 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604445+0000", 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1399999999999998e-06 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638570+0000", 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.034124814000000003 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638586+0000", 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6460000000000002e-05 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638611+0000", 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5216e-05 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638676+0000", 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5048999999999993e-05 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638792+0000", 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115233 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.640186+0000", 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013940090000000001 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657756+0000", 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017570240000000001 2024-09-17T23:30:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679079+0000", 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.021323312000000001 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679150+0000", 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1086000000000003e-05 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679209+0000", 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8284e-05 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.205 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:03.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:03.208 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:03.204+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:03.208 INFO:teuthology.orchestra.run.smithi139.stdout:2435 punch from 0x63b3600 to 0x63c0c00, (0xd600 bytes) 2024-09-17T23:30:03.208 INFO:teuthology.orchestra.run.smithi139.stdout:2436 write 0x44a6200 thru 0x44afb9c (0x999d bytes) 2024-09-17T23:30:03.208 INFO:teuthology.orchestra.run.smithi139.stdout:2437 read 0xb65ca00 thru 0xb66be66 (0xf467 bytes) 2024-09-17T23:30:03.208 INFO:teuthology.orchestra.run.smithi139.stdout:2438 read 0xc656600 thru 0xc65f669 (0x906a bytes) 2024-09-17T23:30:03.208 INFO:teuthology.orchestra.run.smithi139.stdout:2439 punch from 0x4434600 to 0x443d800, (0x9200 bytes) 2024-09-17T23:30:03.209 INFO:teuthology.orchestra.run.smithi139.stdout:2440 trunc from 0xc9ddc00 to 0x5cff800 2024-09-17T23:30:03.209 INFO:teuthology.orchestra.run.smithi139.stdout:2441 write 0xa661200 thru 0xa670e35 (0xfc36 bytes) 2024-09-17T23:30:03.209 INFO:teuthology.orchestra.run.smithi139.stdout:2442 trunc from 0xa671000 to 0x6312600 2024-09-17T23:30:03.307 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:03.305+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:03.355 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:03.356 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 54, 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:30:01.841721+0000", 2024-09-17T23:30:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:28:57.456516+0000", 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:29:26.866978+0000", 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:03.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:03.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:30:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:30:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:30:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.25, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.25, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.25, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.25, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "53", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 21, 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 53, 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.25, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.25, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.25, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.25, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 50, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:30:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:30:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 36, 2024-09-17T23:30:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:30:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:30:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:30:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 36, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:30:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:30:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:30:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:30:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:30:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:30:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:30:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:30:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:30:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 53, 2024-09-17T23:30:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:30:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:30:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:30:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.495330810546875, 2024-09-17T23:30:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:30:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:30:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:30:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:30:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1870185623", 2024-09-17T23:30:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1870185623", 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1870185623", 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1870185623", 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:30:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:30:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:30:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:30:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:30:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:30:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:30:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:30:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:30:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:30:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:03.445 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:03.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:03.448 INFO:tasks.thrashosds.thrasher:j is {'epoch': 54, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:30:01.841721+0000', 'last_up_change': '2024-09-17T23:28:57.456516+0000', 'last_in_change': '2024-09-17T23:29:26.866978+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 53, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 0.495330810546875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1870185623}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1870185623}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1870185623}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1870185623}]}, 'public_addr': '172.21.15.37:6809/1870185623', 'cluster_addr': '172.21.15.37:6811/1870185623', 'heartbeat_back_addr': '172.21.15.37:6815/1870185623', 'heartbeat_front_addr': '172.21.15.37:6813/1870185623', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:30:03.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:30:03.449 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:03.443+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:03.450 INFO:teuthology.orchestra.run.smithi139.stdout:2444 write 0xea4ca00 thru 0xea5210c (0x570d bytes) 2024-09-17T23:30:03.450 INFO:teuthology.orchestra.run.smithi139.stdout:2445 read 0x1101400 thru 0x110ae0c (0x9a0d bytes) 2024-09-17T23:30:03.450 INFO:teuthology.orchestra.run.smithi139.stdout:2446 write 0x72efe00 thru 0x72f2709 (0x290a bytes) 2024-09-17T23:30:03.450 INFO:teuthology.orchestra.run.smithi139.stdout:2451 read 0xd59fc00 thru 0xd5af075 (0xf476 bytes) 2024-09-17T23:30:03.451 INFO:teuthology.orchestra.run.smithi139.stdout:2457 read 0x5814c00 thru 0x5816621 (0x1a22 bytes) 2024-09-17T23:30:03.451 INFO:teuthology.orchestra.run.smithi139.stdout:2458 read 0x2650600 thru 0x265d0f9 (0xcafa bytes) 2024-09-17T23:30:03.451 INFO:teuthology.orchestra.run.smithi139.stdout:2459 write 0x9b32400 thru 0x9b3c542 (0xa143 bytes) 2024-09-17T23:30:03.451 INFO:teuthology.orchestra.run.smithi139.stdout:2463 write 0x42f0c00 thru 0x42f1cd2 (0x10d3 bytes) 2024-09-17T23:30:03.451 INFO:teuthology.orchestra.run.smithi139.stdout:2464 read 0x2b55000 thru 0x2b5a064 (0x5065 bytes) 2024-09-17T23:30:03.451 INFO:teuthology.orchestra.run.smithi139.stdout:2465 read 0xb924800 thru 0xb9250f8 (0x8f9 bytes) 2024-09-17T23:30:03.452 INFO:teuthology.orchestra.run.smithi139.stdout:2467 read 0xcd7c400 thru 0xcd8b0b2 (0xecb3 bytes) 2024-09-17T23:30:03.452 INFO:teuthology.orchestra.run.smithi139.stdout:2469 write 0xc008800 thru 0xc00c3fe (0x3bff bytes) 2024-09-17T23:30:03.452 INFO:teuthology.orchestra.run.smithi139.stdout:2471 trunc from 0xea52200 to 0x23e7000 2024-09-17T23:30:03.470 INFO:teuthology.orchestra.run.smithi139.stdout:2476 punch from 0x3a1600 to 0x3b0000, (0xea00 bytes) 2024-09-17T23:30:03.478 INFO:teuthology.orchestra.run.smithi139.stdout:2477 trunc from 0x23e7000 to 0x758f800 2024-09-17T23:30:03.510 INFO:teuthology.orchestra.run.smithi139.stdout:2478 write 0xd07a800 thru 0xd0854f8 (0xacf9 bytes) 2024-09-17T23:30:03.511 INFO:teuthology.orchestra.run.smithi139.stdout:2479 write 0x2bc9e00 thru 0x2bd59e9 (0xbbea bytes) 2024-09-17T23:30:03.511 INFO:teuthology.orchestra.run.smithi139.stdout:2481 read 0x641a400 thru 0x6429bc2 (0xf7c3 bytes) 2024-09-17T23:30:03.511 INFO:teuthology.orchestra.run.smithi139.stdout:2482 read 0x58c2a00 thru 0x58c4922 (0x1f23 bytes) 2024-09-17T23:30:03.511 INFO:teuthology.orchestra.run.smithi139.stdout:2484 read 0x51d9400 thru 0x51e4328 (0xaf29 bytes) 2024-09-17T23:30:03.511 INFO:teuthology.orchestra.run.smithi139.stdout:2489 punch from 0x7ee1400 to 0x7eec200, (0xae00 bytes) 2024-09-17T23:30:03.511 INFO:teuthology.orchestra.run.smithi139.stdout:2490 write 0x7feb800 thru 0x7ff78e4 (0xc0e5 bytes) 2024-09-17T23:30:03.512 INFO:teuthology.orchestra.run.smithi139.stdout:2492 trunc from 0xd085600 to 0xc729600 2024-09-17T23:30:03.547 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:03.545+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:03.548 INFO:teuthology.orchestra.run.smithi139.stdout:2495 read 0x7320200 thru 0x7329fc3 (0x9dc4 bytes) 2024-09-17T23:30:03.564 INFO:teuthology.orchestra.run.smithi139.stdout:2497 write 0xee5c600 thru 0xee68365 (0xbd66 bytes) 2024-09-17T23:30:03.567 INFO:teuthology.orchestra.run.smithi139.stdout:2498 write 0xa098800 thru 0xa0a3f57 (0xb758 bytes) 2024-09-17T23:30:03.567 INFO:teuthology.orchestra.run.smithi139.stdout:2501 read 0x5454e00 thru 0x545e66e (0x986f bytes) 2024-09-17T23:30:03.567 INFO:teuthology.orchestra.run.smithi139.stdout:2502 write 0xef1000 thru 0xf00a2c (0xfa2d bytes) 2024-09-17T23:30:03.567 INFO:teuthology.orchestra.run.smithi139.stdout:2503 write 0x449be00 thru 0x44a1b38 (0x5d39 bytes) 2024-09-17T23:30:03.567 INFO:teuthology.orchestra.run.smithi139.stdout:2506 read 0x243c200 thru 0x24420ad (0x5eae bytes) 2024-09-17T23:30:03.579 INFO:teuthology.orchestra.run.smithi139.stdout:2507 trunc from 0xee68400 to 0x4b99000 2024-09-17T23:30:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:03.622 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:03.623 INFO:teuthology.orchestra.run.smithi139.stdout:2508 write 0x4970c00 thru 0x49801f4 (0xf5f5 bytes) 2024-09-17T23:30:03.624 INFO:teuthology.orchestra.run.smithi139.stdout:2510 read 0x12cf800 thru 0x12dae6f (0xb670 bytes) 2024-09-17T23:30:03.624 INFO:teuthology.orchestra.run.smithi139.stdout:2514 read 0x44b1600 thru 0x44b2bf1 (0x15f2 bytes) 2024-09-17T23:30:03.630 INFO:teuthology.orchestra.run.smithi139.stdout:2515 trunc from 0x4b99000 to 0xa14dc00 2024-09-17T23:30:03.648 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:03.646+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:03.660 INFO:teuthology.orchestra.run.smithi139.stdout:2518 write 0xd937400 thru 0xd946350 (0xef51 bytes) 2024-09-17T23:30:03.666 INFO:teuthology.orchestra.run.smithi139.stdout:2519 read 0x2b8000 thru 0x2c65e3 (0xe5e4 bytes) 2024-09-17T23:30:03.667 INFO:teuthology.orchestra.run.smithi139.stdout:2522 read 0x87ea000 thru 0x87f2536 (0x8537 bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2524 write 0x1c86200 thru 0x1c86762 (0x563 bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2526 read 0x7c4d400 thru 0x7c509b6 (0x35b7 bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2527 read 0xb2fce00 thru 0xb2ffe82 (0x3083 bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2530 read 0x5df5e00 thru 0x5e030fd (0xd2fe bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2531 punch from 0x4749e00 to 0x4758200, (0xe400 bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2532 write 0x6fd000 thru 0x6fd06a (0x6b bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2535 read 0x4a05a00 thru 0x4a129d6 (0xcfd7 bytes) 2024-09-17T23:30:03.668 INFO:teuthology.orchestra.run.smithi139.stdout:2536 write 0x7f79200 thru 0x7f85e77 (0xcc78 bytes) 2024-09-17T23:30:03.675 INFO:teuthology.orchestra.run.smithi139.stdout:2537 trunc from 0xd946400 to 0x75d5200 2024-09-17T23:30:03.706 INFO:teuthology.orchestra.run.smithi139.stdout:2540 write 0xe75e00 thru 0xe80c4f (0xae50 bytes) 2024-09-17T23:30:03.708 INFO:teuthology.orchestra.run.smithi139.stdout:2541 write 0x2052600 thru 0x205b2c0 (0x8cc1 bytes) 2024-09-17T23:30:03.708 INFO:teuthology.orchestra.run.smithi139.stdout:2542 read 0x154ba00 thru 0x154cfa0 (0x15a1 bytes) 2024-09-17T23:30:03.708 INFO:teuthology.orchestra.run.smithi139.stdout:2544 write 0x478b800 thru 0x478d534 (0x1d35 bytes) 2024-09-17T23:30:03.708 INFO:teuthology.orchestra.run.smithi139.stdout:2545 punch from 0x2a19600 to 0x2a28e00, (0xf800 bytes) 2024-09-17T23:30:03.711 INFO:teuthology.orchestra.run.smithi139.stdout:2547 read 0x607a800 thru 0x608260e (0x7e0f bytes) 2024-09-17T23:30:03.711 INFO:teuthology.orchestra.run.smithi139.stdout:2549 punch from 0x4712600 to 0x471dc00, (0xb600 bytes) 2024-09-17T23:30:03.711 INFO:teuthology.orchestra.run.smithi139.stdout:2550 clone 20 order 19 su 262144 sc 2 2024-09-17T23:30:03.750 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:03.747+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1984 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1814528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.28714772, 2024-09-17T23:30:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049003441000000002, 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1984 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492103+0000", 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1540000000000001e-06 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492107+0000", 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0249999999999996e-06 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492109+0000", 2024-09-17T23:30:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540902+0000", 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048793087999999998 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540995+0000", 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2534000000000005e-05 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541105+0000", 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000109815 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1985 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1823744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.287136426, 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049142263999999998, 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1985 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493825+0000", 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017121879999999999 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493829+0000", 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2509999999999996e-06 2024-09-17T23:30:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493836+0000", 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000002e-06 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493840+0000", 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3220000000000003e-06 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541141+0000", 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047301395000000003 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541190+0000", 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8819999999999997e-05 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541255+0000", 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4587999999999999e-05 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2107 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1105920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.285274261000001, 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048932067000000003, 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2107 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.22e-07 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493977+0000", 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493984+0000", 2024-09-17T23:30:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0920000000000003e-06 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493989+0000", 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0370000000000002e-06 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542786+0000", 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797034000000003 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542841+0000", 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5781e-05 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542907+0000", 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5362000000000002e-05 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2108 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1115136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.285253731000001, 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049148982000000001, 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2108 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493996+0000", 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.03e-07 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493998+0000", 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.187e-06 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494004+0000", 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2219999999999999e-06 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494009+0000", 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6870000000000002e-06 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542954+0000", 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048945206999999998 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543043+0000", 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9494999999999996e-05 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543144+0000", 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100781 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.285233740999999, 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.285213860999999, 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.285192323, 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2117 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1198080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283330008, 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048875710000000003, 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2117 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495920+0000", 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.105e-06 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495924+0000", 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.269e-06 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495926+0000", 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5830000000000001e-06 2024-09-17T23:30:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544618+0000", 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048691940000000003 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544704+0000", 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5581000000000006e-05 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544795+0000", 2024-09-17T23:30:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1020999999999995e-05 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2118 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1207296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283319801000001, 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.04902596, 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2118 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495931+0000", 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.139e-06 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495934+0000", 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1870000000000001e-06 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495936+0000", 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.542e-06 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544842+0000", 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048905954000000001 2024-09-17T23:30:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544893+0000", 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0927999999999998e-05 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544955+0000", 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999000000000001e-05 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283309724999999, 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283299609, 2024-09-17T23:30:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283287379000001, 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.28327681, 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283266921999999, 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283256640000001, 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.283240753000001, 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2129 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1308672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.281332842000001, 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048890621000000002, 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2129 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5100000000000001e-07 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497918+0000", 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4670000000000001e-06 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497922+0000", 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:30:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497924+0000", 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6390000000000001e-06 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546622+0000", 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048697427000000001 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546706+0000", 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4306000000000002e-05 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546807+0000", 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100851 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.7443154359999999, 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.7422460500000003, 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:03.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.4103786820000002, 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:03.879 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:03.881 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:03.882 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:30:03.882 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:03.883 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:03.878+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:03.884 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-17T23:30:02.370799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":701916852,"num_objects":610,"num_object_clones":189,"num_object_copies":1220,"num_objects_missing_on_primary":0,"num_objects_missing":46,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":204,"num_read":136020,"num_read_kb":412677,"num_write":5667,"num_write_kb":238399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":5183154,"num_keys_recovered":29,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5071,"ondisk_log_size":5071,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":230052,"kb_used_data":17972,"kb_used_omap":9,"kb_used_meta":211958,"kb_avail":566000988,"statfs":{"total":579820584960,"available":579585011712,"internally_reserved":0,"allocated":18403328,"data_stored":122312821,"data_compressed":4609836,"data_compressed_allocated":10878976,"data_compressed_original":118757376,"omap_allocated":9566,"internal_metadata":217045666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":116908702,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":46,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":21434,"num_read_kb":114312,"num_write":708,"num_write_kb":145418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000720"},"pg_stats":[{"pgid":"3.1f","version":"49'220","reported_seq":2843,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027045+0000","last_change":"2024-09-17T23:29:16.849137+0000","last_active":"2024-09-17T23:29:58.027045+0000","last_peered":"2024-09-17T23:29:58.027045+0000","last_clean":"2024-09-17T23:29:58.027045+0000","last_became_active":"2024-09-17T23:29:15.605286+0000","last_became_peered":"2024-09-17T23:29:15.605286+0000","last_unstale":"2024-09-17T23:29:58.027045+0000","last_undegraded":"2024-09-17T23:29:58.027045+0000","last_fullsized":"2024-09-17T23:29:58.027045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:55:35.985849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055557299999999996,"stat_sum":{"num_bytes":30495796,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":3574,"num_read_kb":11269,"num_write":253,"num_write_kb":3842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"49'193","reported_seq":4432,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020734+0000","last_change":"2024-09-17T23:29:27.994821+0000","last_active":"2024-09-17T23:29:58.020734+0000","last_peered":"2024-09-17T23:29:58.020734+0000","last_clean":"2024-09-17T23:29:58.020734+0000","last_became_active":"2024-09-17T23:29:27.894511+0000","last_became_peered":"2024-09-17T23:29:27.894511+0000","last_unstale":"2024-09-17T23:29:58.020734+0000","last_undegraded":"2024-09-17T23:29:58.020734+0000","last_fullsized":"2024-09-17T23:29:58.020734+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:48:48.395156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055521100000000001,"stat_sum":{"num_bytes":76732125,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":4641,"num_read_kb":27933,"num_write":269,"num_write_kb":8070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":4447268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"49'78","reported_seq":1228,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027137+0000","last_change":"2024-09-17T23:29:33.928943+0000","last_active":"2024-09-17T23:29:58.027137+0000","last_peered":"2024-09-17T23:29:58.027137+0000","last_clean":"2024-09-17T23:29:58.027137+0000","last_became_active":"2024-09-17T23:29:33.928547+0000","last_became_peered":"2024-09-17T23:29:33.928547+0000","last_unstale":"2024-09-17T23:29:58.027137+0000","last_undegraded":"2024-09-17T23:29:58.027137+0000","last_fullsized":"2024-09-17T23:29:58.027137+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:48:41.148062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476274,"stat_sum":{"num_bytes":15611392,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":1189,"num_read_kb":8284,"num_write":71,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":187904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"49'84","reported_seq":5578,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027858+0000","last_change":"2024-09-17T23:29:39.959779+0000","last_active":"2024-09-17T23:29:58.027858+0000","last_peered":"2024-09-17T23:29:58.027858+0000","last_clean":"2024-09-17T23:29:58.027858+0000","last_became_active":"2024-09-17T23:29:39.959404+0000","last_became_peered":"2024-09-17T23:29:39.959404+0000","last_unstale":"2024-09-17T23:29:58.027858+0000","last_undegraded":"2024-09-17T23:29:58.027858+0000","last_fullsized":"2024-09-17T23:29:58.027858+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:40.703766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263599,"stat_sum":{"num_bytes":9994256,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":5427,"num_read_kb":11335,"num_write":70,"num_write_kb":1494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"49'118","reported_seq":13860,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020402+0000","last_change":"2024-09-17T23:29:16.844175+0000","last_active":"2024-09-17T23:29:58.020402+0000","last_peered":"2024-09-17T23:29:58.020402+0000","last_clean":"2024-09-17T23:29:58.020402+0000","last_became_active":"2024-09-17T23:29:15.601606+0000","last_became_peered":"2024-09-17T23:29:15.601606+0000","last_unstale":"2024-09-17T23:29:58.020402+0000","last_undegraded":"2024-09-17T23:29:58.020402+0000","last_fullsized":"2024-09-17T23:29:58.020402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:21.995818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046809400000000001,"stat_sum":{"num_bytes":49987857,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":13760,"num_read_kb":27791,"num_write":127,"num_write_kb":17614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"49'47","reported_seq":2,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-17T23:29:57.542197+0000","last_change":"2024-09-17T23:29:58.386982+0000","last_active":"2024-09-17T23:29:58.386982+0000","last_peered":"2024-09-17T23:29:57.542197+0000","last_clean":"2024-09-17T23:29:57.542197+0000","last_became_active":"2024-09-17T23:29:39.959168+0000","last_became_peered":"2024-09-17T23:29:39.959168+0000","last_unstale":"2024-09-17T23:29:57.542197+0000","last_undegraded":"2024-09-17T23:29:57.542197+0000","last_fullsized":"2024-09-17T23:29:57.542197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001041527,"stat_sum":{"num_bytes":21846051,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":23,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1873,"num_read_kb":5275,"num_write":45,"num_write_kb":450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"49'158","reported_seq":2012,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024792+0000","last_change":"2024-09-17T23:29:16.855904+0000","last_active":"2024-09-17T23:29:58.024792+0000","last_peered":"2024-09-17T23:29:58.024792+0000","last_clean":"2024-09-17T23:29:58.024792+0000","last_became_active":"2024-09-17T23:29:15.602040+0000","last_became_peered":"2024-09-17T23:29:15.602040+0000","last_unstale":"2024-09-17T23:29:58.024792+0000","last_undegraded":"2024-09-17T23:29:58.024792+0000","last_fullsized":"2024-09-17T23:29:58.024792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:56:09.249189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001306843,"stat_sum":{"num_bytes":12962831,"num_objects":16,"num_object_clones":6,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2747,"num_read_kb":3005,"num_write":125,"num_write_kb":815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"49'214","reported_seq":2017,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020325+0000","last_change":"2024-09-17T23:29:16.844092+0000","last_active":"2024-09-17T23:29:58.020325+0000","last_peered":"2024-09-17T23:29:58.020325+0000","last_clean":"2024-09-17T23:29:58.020325+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:29:58.020325+0000","last_undegraded":"2024-09-17T23:29:58.020325+0000","last_fullsized":"2024-09-17T23:29:58.020325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:07:05.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045869699999999999,"stat_sum":{"num_bytes":17113637,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":2645,"num_read_kb":13821,"num_write":250,"num_write_kb":17183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"49'256","reported_seq":3773,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027146+0000","last_change":"2024-09-17T23:29:16.849195+0000","last_active":"2024-09-17T23:29:58.027146+0000","last_peered":"2024-09-17T23:29:58.027146+0000","last_clean":"2024-09-17T23:29:58.027146+0000","last_became_active":"2024-09-17T23:29:15.601179+0000","last_became_peered":"2024-09-17T23:29:15.601179+0000","last_unstale":"2024-09-17T23:29:58.027146+0000","last_undegraded":"2024-09-17T23:29:58.027146+0000","last_fullsized":"2024-09-17T23:29:58.027146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:32.257776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050274800000000002,"stat_sum":{"num_bytes":32217178,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":5019,"num_read_kb":6252,"num_write":266,"num_write_kb":5982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"46'113","reported_seq":1788,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020592+0000","last_change":"2024-09-17T23:29:33.926843+0000","last_active":"2024-09-17T23:29:58.020592+0000","last_peered":"2024-09-17T23:29:58.020592+0000","last_clean":"2024-09-17T23:29:58.020592+0000","last_became_active":"2024-09-17T23:29:33.926463+0000","last_became_peered":"2024-09-17T23:29:33.926463+0000","last_unstale":"2024-09-17T23:29:58.020592+0000","last_undegraded":"2024-09-17T23:29:58.020592+0000","last_fullsized":"2024-09-17T23:29:58.020592+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:34:47.932725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099808899999999996,"stat_sum":{"num_bytes":12459568,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2386,"num_read_kb":12303,"num_write":113,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":42544,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"49'158","reported_seq":8990,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027032+0000","last_change":"2024-09-17T23:29:16.851584+0000","last_active":"2024-09-17T23:29:58.027032+0000","last_peered":"2024-09-17T23:29:58.027032+0000","last_clean":"2024-09-17T23:29:58.027032+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:29:58.027032+0000","last_undegraded":"2024-09-17T23:29:58.027032+0000","last_fullsized":"2024-09-17T23:29:58.027032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:02:31.853718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073596200000000005,"stat_sum":{"num_bytes":25166888,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":9173,"num_read_kb":11188,"num_write":204,"num_write_kb":2155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"50'197","reported_seq":1429,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027507+0000","last_change":"2024-09-17T23:29:28.004020+0000","last_active":"2024-09-17T23:29:58.027507+0000","last_peered":"2024-09-17T23:29:58.027507+0000","last_clean":"2024-09-17T23:29:58.027507+0000","last_became_active":"2024-09-17T23:29:27.894409+0000","last_became_peered":"2024-09-17T23:29:27.894409+0000","last_unstale":"2024-09-17T23:29:58.027507+0000","last_undegraded":"2024-09-17T23:29:58.027507+0000","last_fullsized":"2024-09-17T23:29:58.027507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:58.054705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013571150000000001,"stat_sum":{"num_bytes":9240686,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":2254,"num_read_kb":8361,"num_write":217,"num_write_kb":16953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":42496,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.028344+0000","last_change":"2024-09-17T23:29:39.959940+0000","last_active":"2024-09-17T23:29:58.028344+0000","last_peered":"2024-09-17T23:29:58.028344+0000","last_clean":"2024-09-17T23:29:58.028344+0000","last_became_active":"2024-09-17T23:29:39.959465+0000","last_became_peered":"2024-09-17T23:29:39.959465+0000","last_unstale":"2024-09-17T23:29:58.028344+0000","last_undegraded":"2024-09-17T23:29:58.028344+0000","last_fullsized":"2024-09-17T23:29:58.028344+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:35.867019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"49'396","reported_seq":566,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024453+0000","last_change":"2024-09-17T23:29:16.856135+0000","last_active":"2024-09-17T23:29:58.024453+0000","last_peered":"2024-09-17T23:29:58.024453+0000","last_clean":"2024-09-17T23:29:58.024453+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:29:58.024453+0000","last_undegraded":"2024-09-17T23:29:58.024453+0000","last_fullsized":"2024-09-17T23:29:58.024453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:50.900745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001532662,"stat_sum":{"num_bytes":1256535,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1999,"num_read_kb":2774,"num_write":687,"num_write_kb":5379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024547+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:29:58.024547+0000","last_peered":"2024-09-17T23:29:58.024547+0000","last_clean":"2024-09-17T23:29:58.024547+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:29:58.024547+0000","last_undegraded":"2024-09-17T23:29:58.024547+0000","last_fullsized":"2024-09-17T23:29:58.024547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"48'155","reported_seq":6320,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024418+0000","last_change":"2024-09-17T23:29:16.855828+0000","last_active":"2024-09-17T23:29:58.024418+0000","last_peered":"2024-09-17T23:29:58.024418+0000","last_clean":"2024-09-17T23:29:58.024418+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:29:58.024418+0000","last_undegraded":"2024-09-17T23:29:58.024418+0000","last_fullsized":"2024-09-17T23:29:58.024418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:01.685935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012515670000000001,"stat_sum":{"num_bytes":11534336,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":7142,"num_read_kb":17433,"num_write":106,"num_write_kb":19493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024296+0000","last_change":"2024-09-17T23:29:39.958959+0000","last_active":"2024-09-17T23:29:58.024296+0000","last_peered":"2024-09-17T23:29:58.024296+0000","last_clean":"2024-09-17T23:29:58.024296+0000","last_became_active":"2024-09-17T23:29:39.958620+0000","last_became_peered":"2024-09-17T23:29:39.958620+0000","last_unstale":"2024-09-17T23:29:58.024296+0000","last_undegraded":"2024-09-17T23:29:58.024296+0000","last_fullsized":"2024-09-17T23:29:58.024296+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:19.215313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"47'179","reported_seq":4986,"reported_epoch":49,"state":"peering","last_fresh":"2024-09-17T23:29:57.039251+0000","last_change":"2024-09-17T23:29:58.024828+0000","last_active":"2024-09-17T23:29:58.024828+0000","last_peered":"2024-09-17T23:29:57.039251+0000","last_clean":"2024-09-17T23:29:57.039251+0000","last_became_active":"2024-09-17T23:29:15.600130+0000","last_became_peered":"2024-09-17T23:29:15.600130+0000","last_unstale":"2024-09-17T23:29:57.039251+0000","last_undegraded":"2024-09-17T23:29:57.039251+0000","last_fullsized":"2024-09-17T23:29:57.039251+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010554410000000001,"stat_sum":{"num_bytes":9269278,"num_objects":20,"num_object_clones":8,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":3219,"num_read_kb":8165,"num_write":238,"num_write_kb":3258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027339+0000","last_change":"2024-09-17T23:29:08.767780+0000","last_active":"2024-09-17T23:29:58.027339+0000","last_peered":"2024-09-17T23:29:58.027339+0000","last_clean":"2024-09-17T23:29:58.027339+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:29:58.027339+0000","last_undegraded":"2024-09-17T23:29:58.027339+0000","last_fullsized":"2024-09-17T23:29:58.027339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:47:22.262418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"49'64","reported_seq":7380,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020470+0000","last_change":"2024-09-17T23:29:33.928884+0000","last_active":"2024-09-17T23:29:58.020470+0000","last_peered":"2024-09-17T23:29:58.020470+0000","last_clean":"2024-09-17T23:29:58.020470+0000","last_became_active":"2024-09-17T23:29:33.928630+0000","last_became_peered":"2024-09-17T23:29:33.928630+0000","last_unstale":"2024-09-17T23:29:58.020470+0000","last_undegraded":"2024-09-17T23:29:58.020470+0000","last_fullsized":"2024-09-17T23:29:58.020470+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:25.983829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17661472,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":7292,"num_read_kb":13360,"num_write":65,"num_write_kb":2485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"49'99","reported_seq":2736,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.028139+0000","last_change":"2024-09-17T23:29:27.969780+0000","last_active":"2024-09-17T23:29:58.028139+0000","last_peered":"2024-09-17T23:29:58.028139+0000","last_clean":"2024-09-17T23:29:58.028139+0000","last_became_active":"2024-09-17T23:29:27.896108+0000","last_became_peered":"2024-09-17T23:29:27.896108+0000","last_unstale":"2024-09-17T23:29:58.028139+0000","last_undegraded":"2024-09-17T23:29:58.028139+0000","last_fullsized":"2024-09-17T23:29:58.028139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:21:54.223571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000582823,"stat_sum":{"num_bytes":17055232,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":3033,"num_read_kb":9386,"num_write":77,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":219648,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":85,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027852+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:29:58.027852+0000","last_peered":"2024-09-17T23:29:58.027852+0000","last_clean":"2024-09-17T23:29:58.027852+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:29:58.027852+0000","last_undegraded":"2024-09-17T23:29:58.027852+0000","last_fullsized":"2024-09-17T23:29:58.027852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"48'176","reported_seq":4174,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.026858+0000","last_change":"2024-09-17T23:29:27.926562+0000","last_active":"2024-09-17T23:29:58.026858+0000","last_peered":"2024-09-17T23:29:58.026858+0000","last_clean":"2024-09-17T23:29:58.026858+0000","last_became_active":"2024-09-17T23:29:27.893418+0000","last_became_peered":"2024-09-17T23:29:27.893418+0000","last_unstale":"2024-09-17T23:29:58.026858+0000","last_undegraded":"2024-09-17T23:29:58.026858+0000","last_fullsized":"2024-09-17T23:29:58.026858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:08.115632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363521,"stat_sum":{"num_bytes":10143843,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":5183,"num_read_kb":14150,"num_write":188,"num_write_kb":666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":81,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020289+0000","last_change":"2024-09-17T23:29:09.377458+0000","last_active":"2024-09-17T23:29:58.020289+0000","last_peered":"2024-09-17T23:29:58.020289+0000","last_clean":"2024-09-17T23:29:58.020289+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:29:58.020289+0000","last_undegraded":"2024-09-17T23:29:58.020289+0000","last_fullsized":"2024-09-17T23:29:58.020289+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:59.106784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"50'261","reported_seq":3078,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.036837+0000","last_change":"2024-09-17T23:29:30.917501+0000","last_active":"2024-09-17T23:29:58.036837+0000","last_peered":"2024-09-17T23:29:58.036837+0000","last_clean":"2024-09-17T23:29:58.036837+0000","last_became_active":"2024-09-17T23:29:30.917041+0000","last_became_peered":"2024-09-17T23:29:30.917041+0000","last_unstale":"2024-09-17T23:29:58.036837+0000","last_undegraded":"2024-09-17T23:29:58.036837+0000","last_fullsized":"2024-09-17T23:29:58.036837+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:15:21.333928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00105678,"stat_sum":{"num_bytes":26162875,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":4417,"num_read_kb":5669,"num_write":174,"num_write_kb":1242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":86528,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-17T23:29:33.924227+0000","last_change":"2024-09-17T23:29:20.564532+0000","last_active":"2024-09-17T23:29:04.482750+0000","last_peered":"2024-09-17T23:29:04.482750+0000","last_clean":"2024-09-17T23:29:04.482750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:29:33.924227+0000","last_undegraded":"2024-09-17T23:29:33.924227+0000","last_fullsized":"2024-09-17T23:29:33.924227+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"49'160","reported_seq":852,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020305+0000","last_change":"2024-09-17T23:29:27.947652+0000","last_active":"2024-09-17T23:29:58.020305+0000","last_peered":"2024-09-17T23:29:58.020305+0000","last_clean":"2024-09-17T23:29:58.020305+0000","last_became_active":"2024-09-17T23:29:27.894483+0000","last_became_peered":"2024-09-17T23:29:27.894483+0000","last_unstale":"2024-09-17T23:29:58.020305+0000","last_undegraded":"2024-09-17T23:29:58.020305+0000","last_fullsized":"2024-09-17T23:29:58.020305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:55:21.215789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047998399999999998,"stat_sum":{"num_bytes":20474977,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":954,"num_read_kb":10231,"num_write":202,"num_write_kb":18953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":79,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027364+0000","last_change":"2024-09-17T23:29:39.959552+0000","last_active":"2024-09-17T23:29:58.027364+0000","last_peered":"2024-09-17T23:29:58.027364+0000","last_clean":"2024-09-17T23:29:58.027364+0000","last_became_active":"2024-09-17T23:29:39.959224+0000","last_became_peered":"2024-09-17T23:29:39.959224+0000","last_unstale":"2024-09-17T23:29:58.027364+0000","last_undegraded":"2024-09-17T23:29:58.027364+0000","last_fullsized":"2024-09-17T23:29:58.027364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:35.806577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":188,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027440+0000","last_change":"2024-09-17T23:29:39.959742+0000","last_active":"2024-09-17T23:29:58.027440+0000","last_peered":"2024-09-17T23:29:58.027440+0000","last_clean":"2024-09-17T23:29:58.027440+0000","last_became_active":"2024-09-17T23:29:39.959373+0000","last_became_peered":"2024-09-17T23:29:39.959373+0000","last_unstale":"2024-09-17T23:29:58.027440+0000","last_undegraded":"2024-09-17T23:29:58.027440+0000","last_fullsized":"2024-09-17T23:29:58.027440+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:43:46.897381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"49'182","reported_seq":8436,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024312+0000","last_change":"2024-09-17T23:29:27.961774+0000","last_active":"2024-09-17T23:29:58.024312+0000","last_peered":"2024-09-17T23:29:58.024312+0000","last_clean":"2024-09-17T23:29:58.024312+0000","last_became_active":"2024-09-17T23:29:27.893544+0000","last_became_peered":"2024-09-17T23:29:27.893544+0000","last_unstale":"2024-09-17T23:29:58.024312+0000","last_undegraded":"2024-09-17T23:29:58.024312+0000","last_fullsized":"2024-09-17T23:29:58.024312+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:20.027314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001536372,"stat_sum":{"num_bytes":20385006,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":9265,"num_read_kb":20606,"num_write":154,"num_write_kb":2884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46096,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"49'142","reported_seq":3368,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.026900+0000","last_change":"2024-09-17T23:29:16.850836+0000","last_active":"2024-09-17T23:29:58.026900+0000","last_peered":"2024-09-17T23:29:58.026900+0000","last_clean":"2024-09-17T23:29:58.026900+0000","last_became_active":"2024-09-17T23:29:15.602469+0000","last_became_peered":"2024-09-17T23:29:15.602469+0000","last_unstale":"2024-09-17T23:29:58.026900+0000","last_undegraded":"2024-09-17T23:29:58.026900+0000","last_fullsized":"2024-09-17T23:29:58.026900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:57:23.878374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483418,"stat_sum":{"num_bytes":11841573,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":2214,"num_read_kb":14249,"num_write":116,"num_write_kb":18216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"49'91","reported_seq":2624,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027735+0000","last_change":"2024-09-17T23:29:16.851336+0000","last_active":"2024-09-17T23:29:58.027735+0000","last_peered":"2024-09-17T23:29:58.027735+0000","last_clean":"2024-09-17T23:29:58.027735+0000","last_became_active":"2024-09-17T23:29:15.605424+0000","last_became_peered":"2024-09-17T23:29:15.605424+0000","last_unstale":"2024-09-17T23:29:58.027735+0000","last_undegraded":"2024-09-17T23:29:58.027735+0000","last_fullsized":"2024-09-17T23:29:58.027735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:05:57.344666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060532200000000004,"stat_sum":{"num_bytes":8068624,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":2503,"num_read_kb":12270,"num_write":98,"num_write_kb":3125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"49'91","reported_seq":1598,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.020480+0000","last_change":"2024-09-17T23:29:16.844405+0000","last_active":"2024-09-17T23:29:58.020480+0000","last_peered":"2024-09-17T23:29:58.020480+0000","last_clean":"2024-09-17T23:29:58.020480+0000","last_became_active":"2024-09-17T23:29:15.601213+0000","last_became_peered":"2024-09-17T23:29:15.601213+0000","last_unstale":"2024-09-17T23:29:58.020480+0000","last_undegraded":"2024-09-17T23:29:58.020480+0000","last_fullsized":"2024-09-17T23:29:58.020480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:05:26.703466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049268800000000002,"stat_sum":{"num_bytes":20929056,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":1970,"num_read_kb":3463,"num_write":70,"num_write_kb":5398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"49'208","reported_seq":1477,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.028217+0000","last_change":"2024-09-17T23:29:16.851583+0000","last_active":"2024-09-17T23:29:58.028217+0000","last_peered":"2024-09-17T23:29:58.028217+0000","last_clean":"2024-09-17T23:29:58.028217+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:29:58.028217+0000","last_undegraded":"2024-09-17T23:29:58.028217+0000","last_fullsized":"2024-09-17T23:29:58.028217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:27.285411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187900000000002,"stat_sum":{"num_bytes":5063717,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2555,"num_read_kb":6879,"num_write":225,"num_write_kb":2438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"49'94","reported_seq":6227,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.024670+0000","last_change":"2024-09-17T23:29:16.847782+0000","last_active":"2024-09-17T23:29:58.024670+0000","last_peered":"2024-09-17T23:29:58.024670+0000","last_clean":"2024-09-17T23:29:58.024670+0000","last_became_active":"2024-09-17T23:29:15.599701+0000","last_became_peered":"2024-09-17T23:29:15.599701+0000","last_unstale":"2024-09-17T23:29:58.024670+0000","last_undegraded":"2024-09-17T23:29:58.024670+0000","last_fullsized":"2024-09-17T23:29:58.024670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:04:21.739508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095122,"stat_sum":{"num_bytes":25963588,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":6139,"num_read_kb":16733,"num_write":113,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"49'152","reported_seq":4069,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027758+0000","last_change":"2024-09-17T23:29:33.928677+0000","last_active":"2024-09-17T23:29:58.027758+0000","last_peered":"2024-09-17T23:29:58.027758+0000","last_clean":"2024-09-17T23:29:58.027758+0000","last_became_active":"2024-09-17T23:29:33.928254+0000","last_became_peered":"2024-09-17T23:29:33.928254+0000","last_unstale":"2024-09-17T23:29:58.027758+0000","last_undegraded":"2024-09-17T23:29:58.027758+0000","last_fullsized":"2024-09-17T23:29:58.027758+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:28:28.440363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093123200000000004,"stat_sum":{"num_bytes":9836117,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":5153,"num_read_kb":9563,"num_write":121,"num_write_kb":2221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"46'77","reported_seq":2144,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.027629+0000","last_change":"2024-09-17T23:29:16.851833+0000","last_active":"2024-09-17T23:29:58.027629+0000","last_peered":"2024-09-17T23:29:58.027629+0000","last_clean":"2024-09-17T23:29:58.027629+0000","last_became_active":"2024-09-17T23:29:15.606245+0000","last_became_peered":"2024-09-17T23:29:15.606245+0000","last_unstale":"2024-09-17T23:29:58.027629+0000","last_undegraded":"2024-09-17T23:29:58.027629+0000","last_fullsized":"2024-09-17T23:29:58.027629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:12.800294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078560200000000005,"stat_sum":{"num_bytes":6452788,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":2086,"num_read_kb":14758,"num_write":99,"num_write_kb":2632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"48'66","reported_seq":2,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-17T23:29:54.598316+0000","last_change":"2024-09-17T23:29:58.393864+0000","last_active":"2024-09-17T23:29:58.393864+0000","last_peered":"2024-09-17T23:29:54.598316+0000","last_clean":"2024-09-17T23:29:54.598316+0000","last_became_active":"2024-09-17T23:29:39.959278+0000","last_became_peered":"2024-09-17T23:29:39.959278+0000","last_unstale":"2024-09-17T23:29:54.598316+0000","last_undegraded":"2024-09-17T23:29:54.598316+0000","last_fullsized":"2024-09-17T23:29:54.598316+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056375800000000003,"stat_sum":{"num_bytes":15722534,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":23,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":337,"num_read_kb":1165,"num_write":60,"num_write_kb":17226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"49'105","reported_seq":2754,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.026805+0000","last_change":"2024-09-17T23:29:39.959636+0000","last_active":"2024-09-17T23:29:58.026805+0000","last_peered":"2024-09-17T23:29:58.026805+0000","last_clean":"2024-09-17T23:29:58.026805+0000","last_became_active":"2024-09-17T23:29:39.959197+0000","last_became_peered":"2024-09-17T23:29:39.959197+0000","last_unstale":"2024-09-17T23:29:58.026805+0000","last_undegraded":"2024-09-17T23:29:58.026805+0000","last_fullsized":"2024-09-17T23:29:58.026805+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:01.590946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043017799999999997,"stat_sum":{"num_bytes":48525824,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":3431,"num_read_kb":20727,"num_write":98,"num_write_kb":3537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":110592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"46'98","reported_seq":2535,"reported_epoch":50,"state":"peering","last_fresh":"2024-09-17T23:29:58.028426+0000","last_change":"2024-09-17T23:29:58.028426+0000","last_active":"2024-09-17T23:29:58.028426+0000","last_peered":"2024-09-17T23:29:50.332921+0000","last_clean":"2024-09-17T23:29:50.332921+0000","last_became_active":"2024-09-17T23:29:15.601164+0000","last_became_peered":"2024-09-17T23:29:15.601164+0000","last_unstale":"2024-09-17T23:29:58.028426+0000","last_undegraded":"2024-09-17T23:29:58.028426+0000","last_fullsized":"2024-09-17T23:29:58.028426+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034516700000000001,"stat_sum":{"num_bytes":61967360,"num_objects":22,"num_object_clones":9,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":2408,"num_read_kb":15259,"num_write":108,"num_write_kb":5235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"50'319","reported_seq":9241,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T23:29:58.028096+0000","last_change":"2024-09-17T23:29:39.959921+0000","last_active":"2024-09-17T23:29:58.028096+0000","last_peered":"2024-09-17T23:29:58.028096+0000","last_clean":"2024-09-17T23:29:58.028096+0000","last_became_active":"2024-09-17T23:29:39.959472+0000","last_became_peered":"2024-09-17T23:29:39.959472+0000","last_unstale":"2024-09-17T23:29:58.028096+0000","last_undegraded":"2024-09-17T23:29:58.028096+0000","last_fullsized":"2024-09-17T23:29:58.028096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:30.308802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001102308,"stat_sum":{"num_bytes":39183465,"num_objects":35,"num_object_clones":12,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":9916,"num_read_kb":48920,"num_write":497,"num_write_kb":27943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":701326465,"num_objects":606,"num_object_clones":189,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":46,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":606,"num_whiteouts":204,"num_read":135904,"num_read_kb":412577,"num_write":5506,"num_write_kb":235981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":5183154,"num_keys_recovered":29,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15015936,"data_stored":119954690,"data_compressed":4643602,"data_compressed_allocated":10862592,"data_compressed_original":117737472,"omap_allocated":0,"internal_metadata":0},"log_size":4951,"ondisk_log_size":4951,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640271,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2060288,"data_stored":13051470,"data_compressed":425733,"data_compressed_allocated":1093632,"data_compressed_original":12606464,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640271,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37212,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334628,"statfs":{"total":96636764160,"available":96598659072,"internally_reserved":0,"allocated":1912832,"data_stored":11970875,"data_compressed":353539,"data_compressed_allocated":876544,"data_compressed_original":11513344,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640271,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39008,"kb_used_data":3660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332832,"statfs":{"total":96636764160,"available":96596819968,"internally_reserved":0,"allocated":3747840,"data_stored":25430162,"data_compressed":951889,"data_compressed_allocated":2289664,"data_compressed_original":24738816,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640271,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38832,"kb_used_data":3488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333008,"statfs":{"total":96636764160,"available":96597000192,"internally_reserved":0,"allocated":3571712,"data_stored":20571948,"data_compressed":964459,"data_compressed_allocated":2109440,"data_compressed_original":19879424,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640272,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38212,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333628,"statfs":{"total":96636764160,"available":96597635072,"internally_reserved":0,"allocated":2932736,"data_stored":21379562,"data_compressed":768746,"data_compressed_allocated":1785856,"data_compressed_original":20825088,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39432,"kb_used_data":4080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332408,"statfs":{"total":96636764160,"available":96596385792,"internally_reserved":0,"allocated":4177920,"data_stored":29908804,"data_compressed":1145470,"data_compressed_allocated":2723840,"data_compressed_original":29194240,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3608576,"data_stored":29685587,"data_compressed":1145470,"data_compressed_allocated":2723840,"data_compressed_original":29194240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2310144,"data_stored":20565996,"data_compressed":763804,"data_compressed_allocated":1748992,"data_compressed_original":20251648,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3096576,"data_stored":20494429,"data_compressed":1008109,"data_compressed_allocated":2166784,"data_compressed_original":20006400,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":25212348,"data_compressed":951889,"data_compressed_allocated":2289664,"data_compressed_original":24738816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1355776,"data_stored":11753042,"data_compressed":353539,"data_compressed_allocated":876544,"data_compressed_original":11513344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1449984,"data_stored":12243288,"data_compressed":420791,"data_compressed_allocated":1056768,"data_compressed_original":12033024,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:30:03.886 INFO:tasks.thrashosds.thrasher:Setting 3.17 to [5, 4] 2024-09-17T23:30:03.886 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.17', '5', '4'] 2024-09-17T23:30:03.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.17 5 4 2024-09-17T23:30:03.982 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:03.979+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:04.030 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:04.031 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:04.031 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:04.031 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:04.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:30:04.123 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:04.121+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:04.183 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:04.181+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:04.186 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:04.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:04.186 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:04.186 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:04.186 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:04.195 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:04.285 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:04.283+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1357 3.17 3:eddf8349:::rbd_data.1152ac6484f7.0000000000000006:10 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593417992999999, 2024-09-17T23:30:04.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025710974000000001, 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1357 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749699+0000", 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749700+0000", 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749702+0000", 2024-09-17T23:30:04.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9879999999999999e-06 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749708+0000", 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0730000000000001e-06 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749713+0000", 2024-09-17T23:30:04.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1250000000000001e-06 2024-09-17T23:30:04.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775212+0000", 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025499457 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775293+0000", 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0961e-05 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775410+0000", 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000116896 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1372 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593397471999999, 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025690635999999999, 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1372 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:30:04.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749720+0000", 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7899999999999999e-07 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749722+0000", 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1560000000000002e-06 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749729+0000", 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749734+0000", 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8450000000000002e-06 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775202+0000", 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025468409000000001 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775294+0000", 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1453999999999995e-05 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775410+0000", 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000116964 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1373 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.356 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593375381, 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025821230000000001, 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1373 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749742+0000", 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4000000000000002e-07 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749744+0000", 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0729999999999999e-06 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749751+0000", 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4740000000000002e-06 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749756+0000", 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0719999999999999e-06 2024-09-17T23:30:04.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775440+0000", 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025684489000000001 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775490+0000", 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9629999999999997e-05 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775563+0000", 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3052000000000005e-05 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1374 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 167936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.59335448, 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026046814000000001, 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.361 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1374 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749763+0000", 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749765+0000", 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0719999999999998e-06 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749772+0000", 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5690000000000001e-06 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749782+0000", 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7489999999999993e-06 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775609+0000", 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025827815 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775699+0000", 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9202000000000004e-05 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775810+0000", 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011099699999999999 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.365 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1375 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593328994, 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.02628755, 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1375 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749788+0000", 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749789+0000", 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.84e-07 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749790+0000", 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5889999999999999e-06 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749794+0000", 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8310000000000003e-06 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749797+0000", 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6369999999999999e-06 2024-09-17T23:30:04.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775863+0000", 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026065878000000001 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.775958+0000", 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5678000000000001e-05 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776076+0000", 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011765300000000001 2024-09-17T23:30:04.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593314579999999, 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:04.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:04.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593282218000001, 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:04.374 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:04.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:04.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593261035999999, 2024-09-17T23:30:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:04.379 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.380 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:04.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593234473999999, 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:04.384 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:04.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.388 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593212300999999, 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.389 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:04.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:04.393 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593192389, 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:04.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593145713, 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.399 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:04.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:04.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593118965, 2024-09-17T23:30:04.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:04.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593090877, 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.412 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.593070578000001, 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:04.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1744 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.586999971999999, 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025572786, 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1744 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756117+0000", 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756118+0000", 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756120+0000", 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0260000000000002e-06 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756124+0000", 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5340000000000001e-06 2024-09-17T23:30:04.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756128+0000", 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.811e-06 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781601+0000", 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025472859 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781649+0000", 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7641e-05 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781690+0000", 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1467000000000001e-05 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1745 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 823296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:30:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.586982384000001, 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025663729, 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1745 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:30:04.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756135+0000", 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9800000000000004e-07 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756138+0000", 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1519999999999999e-06 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756143+0000", 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8769999999999998e-06 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756148+0000", 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5630000000000002e-06 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781715+0000", 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025566720000000001 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781758+0000", 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3683000000000002e-05 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781799+0000", 2024-09-17T23:30:04.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0235999999999999e-05 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1746 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 905216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.586956108999999, 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025754597000000001, 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.426 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1746 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756161+0000", 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756162+0000", 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7400000000000003e-07 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756164+0000", 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.407e-06 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756169+0000", 2024-09-17T23:30:04.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0030000000000002e-06 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756173+0000", 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9589999999999997e-06 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781831+0000", 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025658045000000001 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781875+0000", 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4280000000000003e-05 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781916+0000", 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0329e-05 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1747 3.1f 3:fb7d4906:::rbd_data.1152ac6484f7.0000000000000039:10 [read 983040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:30:04.430 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.586939307, 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025889276999999999, 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1747 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756178+0000", 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7000000000000001e-07 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756180+0000", 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3540000000000001e-06 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756183+0000", 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3400000000000002e-06 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.756185+0000", 2024-09-17T23:30:04.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.486e-06 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.781947+0000", 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.025761954 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.782018+0000", 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0278999999999996e-05 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.782067+0000", 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9594e-05 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.766272051, 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:04.435 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:04.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:04.441 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:04.442 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:04.443 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T23:30:04.443 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:04.441+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:04.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:04.543 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:04.541+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:04.591 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:04.591 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5133.0:1742 3.10 3:0a7e6581:::rbd_data.11521666c10.0000000000000007:head [read 3637248~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:04.589605+0000", 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00028929900000000002, 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031499000000000002, 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5133", 2024-09-17T23:30:04.592 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3523956131", 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1742 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589605+0000", 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589605+0000", 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:04.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589605+0000", 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.75e-07 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589608+0000", 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3379999999999999e-06 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589614+0000", 2024-09-17T23:30:04.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589617+0000", 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8040000000000002e-06 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589818+0000", 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201213 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:04.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:04.589883+0000", 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4657000000000001e-05 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-17T23:30:04.596 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:04.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:04.647 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:04.645+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:04.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:04.720 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:04.721 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:04.721 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:04.721 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:04.721 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:04.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:04.774 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:04.771+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:04.810 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:04.810 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:04.827 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:04.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:04.845 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:04.844+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:04.849 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:04.849 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:04.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:04.861 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:04.861 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:04.878 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:04.879 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:04.895 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:04.895 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:04.930 INFO:teuthology.orchestra.run.smithi139.stdout:2551 trunc from 0x75d5200 to 0xea35c00 2024-09-17T23:30:04.943 INFO:teuthology.orchestra.run.smithi139.stdout:2554 write 0xe7be000 thru 0xe7c582c (0x782d bytes) 2024-09-17T23:30:04.943 INFO:teuthology.orchestra.run.smithi139.stdout:2555 punch from 0x976c800 to 0x9773000, (0x6800 bytes) 2024-09-17T23:30:04.944 INFO:teuthology.orchestra.run.smithi139.stdout:2556 read 0xba20000 thru 0xba2c431 (0xc432 bytes) 2024-09-17T23:30:04.944 INFO:teuthology.orchestra.run.smithi139.stdout:2557 punch from 0xb988800 to 0xb994000, (0xb800 bytes) 2024-09-17T23:30:04.946 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:04.944+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:04.947 INFO:teuthology.orchestra.run.smithi139.stdout:2560 trunc from 0xea35c00 to 0xc52fa00 2024-09-17T23:30:04.974 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:04.974 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:04.983 INFO:teuthology.orchestra.run.smithi139.stdout:2561 trunc from 0xc52fa00 to 0x26b200 2024-09-17T23:30:04.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:05.002 INFO:teuthology.orchestra.run.smithi139.stdout:2563 read 0x25bc00 thru 0x25d151 (0x1552 bytes) 2024-09-17T23:30:05.009 INFO:teuthology.orchestra.run.smithi139.stdout:2565 punch from 0x17ba00 to 0x189c00, (0xe200 bytes) 2024-09-17T23:30:05.034 INFO:teuthology.orchestra.run.smithi139.stdout:2569 write 0xd328600 thru 0xd329343 (0xd44 bytes) 2024-09-17T23:30:05.044 INFO:teuthology.orchestra.run.smithi139.stdout:2572 trunc from 0xd329400 to 0xaa78a00 2024-09-17T23:30:05.047 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:05.046+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:05.061 INFO:teuthology.orchestra.run.smithi139.stdout:2573 read 0x4760c00 thru 0x47624b5 (0x18b6 bytes) 2024-09-17T23:30:05.062 INFO:teuthology.orchestra.run.smithi139.stdout:2574 write 0x3696a00 thru 0x369fd08 (0x9309 bytes) 2024-09-17T23:30:05.062 INFO:teuthology.orchestra.run.smithi139.stdout:2577 read 0x10bfe00 thru 0x10c505e (0x525f bytes) 2024-09-17T23:30:05.062 INFO:teuthology.orchestra.run.smithi139.stdout:2578 read 0x76f000 thru 0x771b18 (0x2b19 bytes) 2024-09-17T23:30:05.062 INFO:teuthology.orchestra.run.smithi139.stdout:2579 read 0x8d8fc00 thru 0x8d96f16 (0x7317 bytes) 2024-09-17T23:30:05.067 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.17 pg_upmap mapping to [5,4] 2024-09-17T23:30:05.071 INFO:teuthology.orchestra.run.smithi139.stdout:2580 trunc from 0xaa78a00 to 0x3086400 2024-09-17T23:30:05.144 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:05.145 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:05.149 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:05.147+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:05.153 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:05.250 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:05.247+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:05.305 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:05.306 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:05.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:05.351 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:05.349+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:05.451 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:05.450+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:05.455 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:05.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:05.552 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:05.550+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:05.571 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:05.571 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:05.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:05.654 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:05.652+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:05.726 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:05.727 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:05.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:05.780 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:05.778+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:05.855 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:05.853+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:05.858 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:05.859 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:05.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:05.956 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:05.954+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:06.016 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.016 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:06.069 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:06.067+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:06.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.133 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:06.183 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:06.181+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:06.258 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:06.257+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:06.289 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.289 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.298 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:06.360 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:06.358+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:06.405 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.405 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:06.498 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:06.496+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:06.561 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.562 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.562 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:06.560+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:06.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:06.663 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:06.661+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:06.718 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.718 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.726 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:06.764 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:06.762+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:06.845 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:30:06.875 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.875 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:06.904 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:06.902+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:06.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:06.965 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:06.963+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:06.990 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:06.990 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:06.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:07.039 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:07.039 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:07.056 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:07.056 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:07.066 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:07.065+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:07.074 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:07.074 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:07.093 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:07.093 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:07.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:07.110 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:07.113 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:07.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:07.113 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:07.113 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:07.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:07.127 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:07.127 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:07.155 INFO:teuthology.orchestra.run.smithi139.stdout:2583 write 0xedd7e00 thru 0xede423d (0xc43e bytes) 2024-09-17T23:30:07.162 INFO:teuthology.orchestra.run.smithi139.stdout:2590 write 0xaa93200 thru 0xaaa1660 (0xe461 bytes) 2024-09-17T23:30:07.162 INFO:teuthology.orchestra.run.smithi139.stdout:2592 write 0x95afc00 thru 0x95be2fe (0xe6ff bytes) 2024-09-17T23:30:07.162 INFO:teuthology.orchestra.run.smithi139.stdout:2594 read 0xbb71a00 thru 0xbb7ff42 (0xe543 bytes) 2024-09-17T23:30:07.162 INFO:teuthology.orchestra.run.smithi139.stdout:2595 read 0xccd7600 thru 0xcce01ce (0x8bcf bytes) 2024-09-17T23:30:07.162 INFO:teuthology.orchestra.run.smithi139.stdout:2598 punch from 0xb398a00 to 0xb3a4c00, (0xc200 bytes) 2024-09-17T23:30:07.172 INFO:teuthology.orchestra.run.smithi139.stdout:2601 trunc from 0xede4400 to 0xa30a800 2024-09-17T23:30:07.179 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:07.177+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:07.200 INFO:teuthology.orchestra.run.smithi139.stdout:2604 trunc from 0xa30a800 to 0xb9c3600 2024-09-17T23:30:07.223 INFO:teuthology.orchestra.run.smithi139.stdout:2605 trunc from 0xb9c3600 to 0x44b0c00 2024-09-17T23:30:07.237 INFO:teuthology.orchestra.run.smithi139.stdout:2606 punch from 0x30da400 to 0x30de800, (0x4400 bytes) 2024-09-17T23:30:07.248 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:07.249 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:07.251 INFO:teuthology.orchestra.run.smithi139.stdout:2609 write 0x6726e00 thru 0x67350f6 (0xe2f7 bytes) 2024-09-17T23:30:07.252 INFO:teuthology.orchestra.run.smithi139.stdout:2610 write 0x3efb000 thru 0x3f03a91 (0x8a92 bytes) 2024-09-17T23:30:07.252 INFO:teuthology.orchestra.run.smithi139.stdout:2613 read 0x345800 thru 0x34d8e0 (0x80e1 bytes) 2024-09-17T23:30:07.252 INFO:teuthology.orchestra.run.smithi139.stdout:2619 read 0x1afdc00 thru 0x1b0b2ac (0xd6ad bytes) 2024-09-17T23:30:07.253 INFO:teuthology.orchestra.run.smithi139.stdout:2621 read 0x306ee00 thru 0x30732bf (0x44c0 bytes) 2024-09-17T23:30:07.253 INFO:teuthology.orchestra.run.smithi139.stdout:2622 punch from 0x6362400 to 0x636cc00, (0xa800 bytes) 2024-09-17T23:30:07.253 INFO:teuthology.orchestra.run.smithi139.stdout:2626 write 0x3001400 thru 0x300a838 (0x9439 bytes) 2024-09-17T23:30:07.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:07.273 INFO:teuthology.orchestra.run.smithi139.stdout:2627 write 0xc8ac600 thru 0xc8bbc34 (0xf635 bytes) 2024-09-17T23:30:07.279 INFO:teuthology.orchestra.run.smithi139.stdout:2628 punch from 0xc083800 to 0xc089000, (0x5800 bytes) 2024-09-17T23:30:07.281 INFO:teuthology.orchestra.run.smithi139.stdout:2630 write 0x4644c00 thru 0x464f559 (0xa95a bytes) 2024-09-17T23:30:07.281 INFO:teuthology.orchestra.run.smithi139.stdout:2631 read 0xa28d200 thru 0xa294bfd (0x79fe bytes) 2024-09-17T23:30:07.292 INFO:teuthology.orchestra.run.smithi139.stdout:2632 trunc from 0xc8bbe00 to 0x3577200 2024-09-17T23:30:07.302 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:07.300+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:07.332 INFO:teuthology.orchestra.run.smithi139.stdout:2633 write 0xceea800 thru 0xcef62e0 (0xbae1 bytes) 2024-09-17T23:30:07.339 INFO:teuthology.orchestra.run.smithi139.stdout:2634 write 0x65cec00 thru 0x65d979f (0xaba0 bytes) 2024-09-17T23:30:07.342 INFO:teuthology.orchestra.run.smithi139.stdout:2636 trunc from 0xcef6400 to 0xa20b800 2024-09-17T23:30:07.360 INFO:teuthology.orchestra.run.smithi139.stdout:2641 read 0x36cc600 thru 0x36d1422 (0x4e23 bytes) 2024-09-17T23:30:07.365 INFO:teuthology.orchestra.run.smithi139.stdout:2642 write 0x9739800 thru 0x973d10b (0x390c bytes) 2024-09-17T23:30:07.366 INFO:teuthology.orchestra.run.smithi139.stdout:2645 write 0x2f0da00 thru 0x2f0f5d1 (0x1bd2 bytes) 2024-09-17T23:30:07.366 INFO:teuthology.orchestra.run.smithi139.stdout:2646 read 0x1f29a00 thru 0x1f35608 (0xbc09 bytes) 2024-09-17T23:30:07.366 INFO:teuthology.orchestra.run.smithi139.stdout:2647 read 0x19f4000 thru 0x19f8e53 (0x4e54 bytes) 2024-09-17T23:30:07.366 INFO:teuthology.orchestra.run.smithi139.stdout:2648 write 0x20a8e00 thru 0x20adfb5 (0x51b6 bytes) 2024-09-17T23:30:07.367 INFO:teuthology.orchestra.run.smithi139.stdout:2649 trunc from 0xa20b800 to 0xba73400 2024-09-17T23:30:07.369 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:07.368+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:07.380 INFO:teuthology.orchestra.run.smithi139.stdout:2653 read 0x9b8fe00 thru 0x9b9fbd5 (0xfdd6 bytes) 2024-09-17T23:30:07.398 INFO:teuthology.orchestra.run.smithi139.stdout:2656 write 0xe9f3200 thru 0xe9f739b (0x419c bytes) 2024-09-17T23:30:07.399 INFO:teuthology.orchestra.run.smithi139.stdout:2657 write 0x7bde200 thru 0x7be3bff (0x5a00 bytes) 2024-09-17T23:30:07.411 INFO:teuthology.orchestra.run.smithi139.stdout:2658 trunc from 0xe9f7400 to 0x423b000 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9550 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.135026321, 2024-09-17T23:30:07.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056192745000000002, 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9550 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290751+0000", 2024-09-17T23:30:07.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290757+0000", 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290762+0000", 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346584+0000", 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055821636000000001 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346864+0000", 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028079899999999998 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346941+0000", 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6173000000000004e-05 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9552 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134789407, 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056146769999999999, 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9552 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.434 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:07.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8999999999999996e-07 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290994+0000", 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3399999999999998e-06 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291001+0000", 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1910000000000003e-06 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291007+0000", 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346974+0000", 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055966828000000003 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347059+0000", 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5753000000000004e-05 2024-09-17T23:30:07.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347131+0000", 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9604 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134720034000001, 2024-09-17T23:30:07.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05626722, 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9604 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291055+0000", 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291057+0000", 2024-09-17T23:30:07.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291064+0000", 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291070+0000", 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347164+0000", 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056094887000000003 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347250+0000", 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5552999999999999e-05 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347321+0000", 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1310999999999995e-05 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9606 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134454342, 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05619619, 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9606 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1900000000000002e-07 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291324+0000", 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6619999999999998e-06 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291333+0000", 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291339+0000", 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.477e-06 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347352+0000", 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056012987 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347434+0000", 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1569000000000005e-05 2024-09-17T23:30:07.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347516+0000", 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2365000000000002e-05 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9608 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134424181, 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056951664999999999, 2024-09-17T23:30:07.447 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9608 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291351+0000", 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8299999999999997e-07 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291353+0000", 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.396e-06 2024-09-17T23:30:07.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291360+0000", 2024-09-17T23:30:07.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.199e-06 2024-09-17T23:30:07.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291367+0000", 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2230000000000001e-06 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347546+0000", 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056178616000000001 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348232+0000", 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068600200000000001 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348302+0000", 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9646000000000004e-05 2024-09-17T23:30:07.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9610 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134261491, 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056957865000000003, 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9610 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291517+0000", 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.393e-06 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291526+0000", 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0129999999999999e-06 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291532+0000", 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8430000000000004e-06 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348331+0000", 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056798279 2024-09-17T23:30:07.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348405+0000", 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4418000000000005e-05 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348470+0000", 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5355e-05 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9613 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134191446999999, 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057056663000000001, 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9613 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291585+0000", 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291593+0000", 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291596+0000", 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.269e-06 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348498+0000", 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056902369000000001 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348573+0000", 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4962000000000005e-05 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348639+0000", 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5874e-05 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9615 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.134027729, 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057044395999999997, 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9615 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291747+0000", 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291749+0000", 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291754+0000", 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410000000000003e-06 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291757+0000", 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4520000000000002e-06 2024-09-17T23:30:07.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348665+0000", 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056908513000000001 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348732+0000", 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6180999999999995e-05 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348791+0000", 2024-09-17T23:30:07.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9129000000000003e-05 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9620 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.133873376, 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057063968999999999, 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9620 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291909+0000", 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5610000000000001e-06 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291916+0000", 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0679999999999999e-06 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291923+0000", 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348817+0000", 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056893959000000001 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348898+0000", 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0377999999999995e-05 2024-09-17T23:30:07.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348965+0000", 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7146999999999999e-05 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9622 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.133814900999999, 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057231199000000003, 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9622 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291960+0000", 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291962+0000", 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7939999999999998e-06 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291970+0000", 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5009999999999998e-06 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291975+0000", 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999999e-06 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349015+0000", 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057039443000000002 2024-09-17T23:30:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349104+0000", 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9791999999999997e-05 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349190+0000", 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6055000000000002e-05 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9716 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.129669002, 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056085545000000001, 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9716 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296106+0000", 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296108+0000", 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296114+0000", 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7830000000000004e-06 2024-09-17T23:30:07.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296119+0000", 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352074+0000", 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055955619999999998 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352121+0000", 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6857999999999998e-05 2024-09-17T23:30:07.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352191+0000", 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9639000000000002e-05 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9722 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.129610083999999, 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056209310999999998, 2024-09-17T23:30:07.482 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9722 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296362+0000", 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019768800000000001 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296366+0000", 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9419999999999997e-06 2024-09-17T23:30:07.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296375+0000", 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6309999999999998e-06 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296392+0000", 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6758000000000001e-05 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352221+0000", 2024-09-17T23:30:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055828950000000002 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352299+0000", 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8375999999999996e-05 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352373+0000", 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3966000000000003e-05 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9723 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.129371175999999, 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056145054999999999, 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9723 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296404+0000", 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8800000000000002e-07 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296411+0000", 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296419+0000", 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129999999999993e-06 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296424+0000", 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5879999999999997e-06 2024-09-17T23:30:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352405+0000", 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055980769 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352479+0000", 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4018999999999997e-05 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352548+0000", 2024-09-17T23:30:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8997000000000004e-05 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9729 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.129315995000001, 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056272003000000001, 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9729 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296459+0000", 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2799999999999996e-07 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296461+0000", 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439999999999998e-06 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296468+0000", 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296472+0000", 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352583+0000", 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056111017999999999 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352665+0000", 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1148000000000003e-05 2024-09-17T23:30:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352730+0000", 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5477999999999999e-05 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9731 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.129294610000001, 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056506531999999998, 2024-09-17T23:30:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9731 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297569+0000", 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001089394 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297574+0000", 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7260000000000002e-06 2024-09-17T23:30:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297583+0000", 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4169999999999998e-06 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297590+0000", 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352775+0000", 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055185161000000003 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352867+0000", 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2090999999999996e-05 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352986+0000", 2024-09-17T23:30:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011883300000000001 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9743 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.128024996000001, 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055972380000000002, 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9743 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297750+0000", 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297752+0000", 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297760+0000", 2024-09-17T23:30:07.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1839999999999999e-06 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297766+0000", 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353551+0000", 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055784783999999997 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353645+0000", 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3760999999999999e-05 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353721+0000", 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6907000000000001e-05 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9744 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.128000128, 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056155391999999998, 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9744 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297775+0000", 2024-09-17T23:30:07.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297777+0000", 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298431+0000", 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065376599999999996 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298441+0000", 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8709999999999992e-06 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353773+0000", 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055332332999999997 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353856+0000", 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3122999999999997e-05 2024-09-17T23:30:07.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353929+0000", 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3183999999999996e-05 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9795 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 4677632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.125616302999999, 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056138625999999997, 2024-09-17T23:30:07.509 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9795 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301567+0000", 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001409492 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301573+0000", 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2159999999999997e-06 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301582+0000", 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5619999999999993e-06 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301588+0000", 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356166+0000", 2024-09-17T23:30:07.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054577696000000002 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356253+0000", 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6526000000000005e-05 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356296+0000", 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3918999999999997e-05 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12612 3.13 3:ccf171c8:::rbd_data.115262cdab6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.91508305, 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063461453000000001, 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12612 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510693+0000", 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9e-06 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539114+0000", 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028420697000000002 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539145+0000", 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1488000000000001e-05 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539159+0000", 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3737000000000001e-05 2024-09-17T23:30:07.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539226+0000", 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7572999999999998e-05 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539337+0000", 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110849 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.540857+0000", 2024-09-17T23:30:07.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001520043 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.571640+0000", 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030782961000000001 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574080+0000", 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024394529999999998 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:07.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574120+0000", 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0459e-05 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574153+0000", 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292999999999997e-05 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12613 3.9 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:07.519 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.886585048000001, 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.13090919500000001, 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12613 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:07.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539191+0000", 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.59e-06 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.595998+0000", 2024-09-17T23:30:07.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056807745999999999 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596021+0000", 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2574000000000001e-05 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596030+0000", 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1670000000000005e-06 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596150+0000", 2024-09-17T23:30:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119688 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596301+0000", 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015097400000000001 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598103+0000", 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001802281 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:07.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.610230+0000", 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012126468 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.669969+0000", 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059739318999999999 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670041+0000", 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1834000000000004e-05 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670098+0000", 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7553999999999998e-05 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:07.525 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:07.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:07.528 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:07.524+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2660 write 0xbaeee00 thru 0xbaf290c (0x3b0d bytes) 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2663 write 0x5215400 thru 0x521ec91 (0x9892 bytes) 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2664 punch from 0x34ea400 to 0x34ed400, (0x3000 bytes) 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2667 write 0xea71000 thru 0xea7dd51 (0xcd52 bytes) 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2668 read 0xbf92400 thru 0xbfa07a6 (0xe3a7 bytes) 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2669 trunc from 0xea7de00 to 0x20c9e00 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2670 write 0x2f95e00 thru 0x2f97785 (0x1986 bytes) 2024-09-17T23:30:07.529 INFO:teuthology.orchestra.run.smithi139.stdout:2674 read 0xdf2400 thru 0xdf6abb (0x46bc bytes) 2024-09-17T23:30:07.530 INFO:teuthology.orchestra.run.smithi139.stdout:2675 trunc from 0x2f97800 to 0x3f2400 2024-09-17T23:30:07.558 INFO:teuthology.orchestra.run.smithi139.stdout:2676 trunc from 0x3f2400 to 0xea14e00 2024-09-17T23:30:07.566 INFO:teuthology.orchestra.run.smithi139.stdout:2677 write 0x320bc00 thru 0x3213558 (0x7959 bytes) 2024-09-17T23:30:07.567 INFO:teuthology.orchestra.run.smithi139.stdout:2680 read 0x2256800 thru 0x225bddd (0x55de bytes) 2024-09-17T23:30:07.567 INFO:teuthology.orchestra.run.smithi139.stdout:2682 punch from 0xc620a00 to 0xc627200, (0x6800 bytes) 2024-09-17T23:30:07.569 INFO:teuthology.orchestra.run.smithi139.stdout:2684 flatten 2024-09-17T23:30:07.588 INFO:teuthology.orchestra.run.smithi139.stdout:2687 read 0x8c7f400 thru 0x8c89bad (0xa7ae bytes) 2024-09-17T23:30:07.590 INFO:teuthology.orchestra.run.smithi139.stdout:2688 read 0x14e0e00 thru 0x14e360b (0x280c bytes) 2024-09-17T23:30:07.590 INFO:teuthology.orchestra.run.smithi139.stdout:2689 punch from 0xaba7a00 to 0xaba7c00, (0x200 bytes) 2024-09-17T23:30:07.590 INFO:teuthology.orchestra.run.smithi139.stdout:2690 write 0x130be00 thru 0x130c7e3 (0x9e4 bytes) 2024-09-17T23:30:07.590 INFO:teuthology.orchestra.run.smithi139.stdout:2692 write 0x7ec1600 thru 0x7ecd0a9 (0xbaaa bytes) 2024-09-17T23:30:07.590 INFO:teuthology.orchestra.run.smithi139.stdout:2693 read 0x65af200 thru 0x65b4c98 (0x5a99 bytes) 2024-09-17T23:30:07.590 INFO:teuthology.orchestra.run.smithi139.stdout:2694 read 0xbd60400 thru 0xbd6679d (0x639e bytes) 2024-09-17T23:30:07.607 INFO:teuthology.orchestra.run.smithi139.stdout:2695 trunc from 0xea14e00 to 0x4d9e00 2024-09-17T23:30:07.627 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:07.625+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:07.629 INFO:teuthology.orchestra.run.smithi139.stdout:2698 read 0xee200 thru 0xfa856 (0xc657 bytes) 2024-09-17T23:30:07.634 INFO:teuthology.orchestra.run.smithi139.stdout:2700 punch from 0x9b600 to 0xa7e00, (0xc800 bytes) 2024-09-17T23:30:07.634 INFO:teuthology.orchestra.run.smithi139.stdout:2701 read 0x129600 thru 0x1323d8 (0x8dd9 bytes) 2024-09-17T23:30:07.650 INFO:teuthology.orchestra.run.smithi139.stdout:2704 write 0x3e15200 thru 0x3e18fb2 (0x3db3 bytes) 2024-09-17T23:30:07.651 INFO:teuthology.orchestra.run.smithi139.stdout:2705 trunc from 0x3e19000 to 0x2e8dc00 2024-09-17T23:30:07.667 INFO:teuthology.orchestra.run.smithi139.stdout:2710 write 0x3f800 thru 0x3f930 (0x131 bytes) 2024-09-17T23:30:07.670 INFO:teuthology.orchestra.run.smithi139.stdout:2711 read 0x2019a00 thru 0x202085e (0x6e5f bytes) 2024-09-17T23:30:07.672 INFO:teuthology.orchestra.run.smithi139.stdout:2712 trunc from 0x2e8dc00 to 0x913200 2024-09-17T23:30:07.689 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:07.689 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:07.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:07.709 INFO:teuthology.orchestra.run.smithi139.stdout:2713 write 0xd1a8600 thru 0xd1b0de0 (0x87e1 bytes) 2024-09-17T23:30:07.710 INFO:teuthology.orchestra.run.smithi139.stdout:2716 read 0x12dc00 thru 0x13a602 (0xca03 bytes) 2024-09-17T23:30:07.719 INFO:teuthology.orchestra.run.smithi139.stdout:2723 trunc from 0xd1b0e00 to 0x7e29e00 2024-09-17T23:30:07.742 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:07.740+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:07.750 INFO:teuthology.orchestra.run.smithi139.stdout:2724 write 0x9908000 thru 0x99102ba (0x82bb bytes) 2024-09-17T23:30:07.750 INFO:teuthology.orchestra.run.smithi139.stdout:2725 read 0x467aa00 thru 0x4687e55 (0xd456 bytes) 2024-09-17T23:30:07.750 INFO:teuthology.orchestra.run.smithi139.stdout:2727 write 0x9777e00 thru 0x9778a45 (0xc46 bytes) 2024-09-17T23:30:07.751 INFO:teuthology.orchestra.run.smithi139.stdout:2728 write 0x8377800 thru 0x838150d (0x9d0e bytes) 2024-09-17T23:30:07.753 INFO:teuthology.orchestra.run.smithi139.stdout:2736 trunc from 0x9910400 to 0x849d400 2024-09-17T23:30:07.771 INFO:teuthology.orchestra.run.smithi139.stdout:2738 write 0x6ea2600 thru 0x6ea350b (0xf0c bytes) 2024-09-17T23:30:07.796 INFO:teuthology.orchestra.run.smithi139.stdout:2739 write 0xd2ff200 thru 0xd30a4d8 (0xb2d9 bytes) 2024-09-17T23:30:07.797 INFO:teuthology.orchestra.run.smithi139.stdout:2741 read 0x2b94c00 thru 0x2b9b8ee (0x6cef bytes) 2024-09-17T23:30:07.802 INFO:teuthology.orchestra.run.smithi139.stdout:2744 trunc from 0xd30a600 to 0x9016000 2024-09-17T23:30:07.819 INFO:teuthology.orchestra.run.smithi139.stdout:2746 write 0x1938e00 thru 0x193c5b6 (0x37b7 bytes) 2024-09-17T23:30:07.819 INFO:teuthology.orchestra.run.smithi139.stdout:2747 read 0x394b400 thru 0x3955600 (0xa201 bytes) 2024-09-17T23:30:07.824 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:07.824 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:07.824 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:07.824 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:07.825 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:07.829 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:07.827+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:07.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:07.838 INFO:teuthology.orchestra.run.smithi139.stdout:2748 write 0xd777c00 thru 0xd778c26 (0x1027 bytes) 2024-09-17T23:30:07.840 INFO:teuthology.orchestra.run.smithi139.stdout:2750 read 0xaaee00 thru 0xab97fd (0xa9fe bytes) 2024-09-17T23:30:07.840 INFO:teuthology.orchestra.run.smithi139.stdout:2751 write 0x79a1000 thru 0x79af8be (0xe8bf bytes) 2024-09-17T23:30:07.840 INFO:teuthology.orchestra.run.smithi139.stdout:2752 read 0xcd87200 thru 0xcd95b2c (0xe92d bytes) 2024-09-17T23:30:07.840 INFO:teuthology.orchestra.run.smithi139.stdout:2753 read 0x4c33000 thru 0x4c3e66a (0xb66b bytes) 2024-09-17T23:30:07.840 INFO:teuthology.orchestra.run.smithi139.stdout:2754 write 0xb5e7000 thru 0xb5f6044 (0xf045 bytes) 2024-09-17T23:30:07.853 INFO:teuthology.orchestra.run.smithi139.stdout:2755 trunc from 0xd778e00 to 0x2882800 2024-09-17T23:30:07.881 INFO:teuthology.orchestra.run.smithi139.stdout:2757 trunc from 0x2882800 to 0x3fbe200 2024-09-17T23:30:07.891 INFO:teuthology.orchestra.run.smithi139.stdout:2762 read 0x1bc0e00 thru 0x1bc1895 (0xa96 bytes) 2024-09-17T23:30:07.913 INFO:teuthology.orchestra.run.smithi139.stdout:2765 write 0xcd88600 thru 0xcd8f90a (0x730b bytes) 2024-09-17T23:30:07.914 INFO:teuthology.orchestra.run.smithi139.stdout:2768 read 0x92f400 thru 0x930d7d (0x197e bytes) 2024-09-17T23:30:07.914 INFO:teuthology.orchestra.run.smithi139.stdout:2769 punch from 0x4083200 to 0x4091c00, (0xea00 bytes) 2024-09-17T23:30:07.927 INFO:teuthology.orchestra.run.smithi139.stdout:2770 trunc from 0xcd8fa00 to 0x307de00 2024-09-17T23:30:07.929 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:07.927+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:07.955 INFO:teuthology.orchestra.run.smithi139.stdout:2771 write 0x4ccfc00 thru 0x4cdddde (0xe1df bytes) 2024-09-17T23:30:07.956 INFO:teuthology.orchestra.run.smithi139.stdout:2772 read 0x233a200 thru 0x23467d9 (0xc5da bytes) 2024-09-17T23:30:07.971 INFO:teuthology.orchestra.run.smithi139.stdout:2773 write 0x6041000 thru 0x6049291 (0x8292 bytes) 2024-09-17T23:30:07.971 INFO:teuthology.orchestra.run.smithi139.stdout:2775 read 0xae0800 thru 0xae602b (0x582c bytes) 2024-09-17T23:30:07.971 INFO:teuthology.orchestra.run.smithi139.stdout:2776 write 0x2a5f400 thru 0x2a5f9a6 (0x5a7 bytes) 2024-09-17T23:30:07.978 INFO:teuthology.orchestra.run.smithi139.stdout:2777 trunc from 0x6049400 to 0xbe0d000 2024-09-17T23:30:07.997 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.523464082000004, 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.665628269000001, 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.664935976000001, 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:08.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:08.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.664924395, 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:08.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.664397148000001, 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:08.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:08.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.664363164999999, 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:08.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:08.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.664052572999999, 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.030 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.664026439000001, 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:08.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.663528532000001, 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:08.039 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:08.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.662927442000001, 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:08.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.662378807, 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:08.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.662171209, 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:08.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.652555308, 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:08.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.652545074000001, 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:08.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:08.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.652535207, 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.652525059, 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:08.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.652055191000001, 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:08.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.652033075, 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:08.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.651745379999999, 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:08.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:08.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.46635921, 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:08.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:08.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.104 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:08.105 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:08.103+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:08.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2778 write 0x5942c00 thru 0x59442b3 (0x16b4 bytes) 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2779 write 0x1a56600 thru 0x1a5771e (0x111f bytes) 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2780 write 0x8339400 thru 0x833ce54 (0x3a55 bytes) 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2783 punch from 0x102fc00 to 0x1039c00, (0xa000 bytes) 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2784 trunc from 0xbe0d000 to 0xd9a7800 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2790 trunc from 0xd9a7800 to 0x95a9c00 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2792 read 0x4a73000 thru 0x4a7337d (0x37e bytes) 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2793 read 0x5684800 thru 0x5689b7a (0x537b bytes) 2024-09-17T23:30:08.108 INFO:teuthology.orchestra.run.smithi139.stdout:2794 write 0xa7d3e00 thru 0xa7da290 (0x6491 bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2795 trunc from 0xa7da400 to 0xbed6800 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2797 read 0x2162000 thru 0x216dfe3 (0xbfe4 bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2798 write 0xc85a800 thru 0xc85ca82 (0x2283 bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2800 write 0x3e05600 thru 0x3e08512 (0x2f13 bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2804 read 0xb9ca00 thru 0xba2c79 (0x627a bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2805 read 0x6d14c00 thru 0x6d2448f (0xf890 bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2806 read 0x8db6200 thru 0x8dbed43 (0x8b44 bytes) 2024-09-17T23:30:08.109 INFO:teuthology.orchestra.run.smithi139.stdout:2810 trunc from 0xc85cc00 to 0xa992c00 2024-09-17T23:30:08.206 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:08.205+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:08.216 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:08.216 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:08.216 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:08.216 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:08.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:08.307 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:08.306+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:08.371 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:08.372 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:08.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:08.424 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:08.422+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:08.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14760 3.11 3:8ddf073a:::rbd_data.1152c1142907.0000000000000029:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.452416347, 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010811016999999999, 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14760 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034622+0000", 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034623+0000", 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034627+0000", 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4680000000000003e-06 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034635+0000", 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.034641+0000", 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035590+0000", 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094941100000000005 2024-09-17T23:30:08.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036421+0000", 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083071299999999996 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.036612+0000", 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191342 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041698+0000", 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050861980000000001 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045354+0000", 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036561440000000001 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045394+0000", 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9604999999999998e-05 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.045433+0000", 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8945000000000001e-05 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.451382412000001, 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:08.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:08.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:08.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.451209471999999, 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:08.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:08.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:08.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.451169602, 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:08.513 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:08.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:08.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.156665374999999, 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:08.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:08.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.908963826000001, 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:08.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.899031559999999, 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.885804511, 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:289 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3452983189999994, 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010772220000000001, 2024-09-17T23:30:08.541 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 289 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141740+0000", 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:08.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141743+0000", 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141750+0000", 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-06 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141755+0000", 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1510000000000003e-06 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142121+0000", 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036654199999999997 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142125+0000", 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.968e-06 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152136+0000", 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010011124 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152323+0000", 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018617699999999999 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152512+0000", 2024-09-17T23:30:08.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001896 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3452753550000001, 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:08.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:08.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3452539649999995, 2024-09-17T23:30:08.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:08.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:08.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:08.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3452368359999998, 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:08.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3452195539999998, 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:08.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3452025249999995, 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:08.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:08.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3451696510000009, 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:08.575 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:08.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:08.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:08.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3445819449999998, 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:08.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:08.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:08.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:08.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3227898040000001, 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:08.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3222526160000001, 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:08.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:08.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:08.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:08.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:08.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:08.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:08.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:08.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:08.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:08.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:08.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:08.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:08.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:08.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3217665000000001, 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:08.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:08.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:709 3.3 e58/30 3:c0ea72f3:::rbd_data.1152ce7c866.00000000000001db:head v 58'120, mlcod=54'118)", 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.166806+0000", 2024-09-17T23:30:08.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.320232313, 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011018765, 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166806+0000", 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166806+0000", 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166807+0000", 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2799999999999995e-07 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166847+0000", 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0488999999999998e-05 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166859+0000", 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1664999999999999e-05 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166864+0000", 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166924+0000", 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0214000000000003e-05 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.166946+0000", 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1345999999999999e-05 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177783+0000", 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01083771 2024-09-17T23:30:08.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177825+0000", 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1498000000000001e-05 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:08.612 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:08.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:08.614 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:08.612+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:08.714 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:08.712+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:08.730 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:08.730 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:08.730 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:08.730 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:08.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:08.815 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:08.813+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:08.847 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:08.847 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:08.847 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:08.847 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:08.848 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:08.857 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:08.863 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:30:08.941 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:08.939+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:09.007 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:09.007 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:09.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:09.007 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:09.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.007 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.129161146999998, 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.008 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:09.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:09.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113536017000001, 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:09.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113498360999998, 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.020 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.025 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113480930000001, 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:09.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:09.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:09.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113445999, 2024-09-17T23:30:09.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:09.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113414579000001, 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:09.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:09.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:09.041 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113397953000003, 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.042 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:09.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:09.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113376258999999, 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.047 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:09.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:09.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 42.113364541000003, 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:09.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.549162198000001, 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:30:09.057 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:09.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:30:09.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.549139732, 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:09.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3262 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.541694953, 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045783590999999998, 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3262 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463817+0000", 2024-09-17T23:30:09.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00160411 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463821+0000", 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6670000000000004e-06 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463838+0000", 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6898999999999999e-05 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463848+0000", 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507787+0000", 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043939260000000001 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507895+0000", 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010859899999999999 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507996+0000", 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100587 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.531801448, 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:30:09.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.529893554000001, 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:30:09.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:30:09.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:30:09.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:09.080 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.527908484000001, 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:09.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:30:09.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.527898888999999, 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:30:09.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12611 3.18 3:1c8edc2b:::rbd_data.115262cdab6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:09.089 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 15.494588169, 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.070059079999999996, 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12611 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509321+0000", 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-06 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553867+0000", 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044545920000000003 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553893+0000", 2024-09-17T23:30:09.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6094000000000001e-05 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553906+0000", 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3079e-05 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553972+0000", 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6630000000000004e-05 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.554090+0000", 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117364 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.555383+0000", 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001293188 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.577545+0000", 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.022161943 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579298+0000", 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017536349999999999 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579346+0000", 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7858000000000002e-05 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579378+0000", 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2011000000000002e-05 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.098 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12614 3.2 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 15.449967386999999, 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.100611516, 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:09.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12614 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553942+0000", 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.477e-06 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599428+0000", 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045486208 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599454+0000", 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6551999999999999e-05 2024-09-17T23:30:09.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599467+0000", 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2339e-05 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599548+0000", 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599660+0000", 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011209699999999999 2024-09-17T23:30:09.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.601086+0000", 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014260550000000001 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.612776+0000", 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011690010000000001 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654410+0000", 2024-09-17T23:30:09.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.041634517000000003 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654486+0000", 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6260999999999994e-05 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654552+0000", 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5086999999999995e-05 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4968.0:12613 3.9 e48/19 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head v 48'120, mlcod=45'117)", 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 15.405709292999999, 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049712148999999997, 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598199+0000", 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-07 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637181+0000", 2024-09-17T23:30:09.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.038982528000000002 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637209+0000", 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7317e-05 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637219+0000", 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0372e-05 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637307+0000", 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7867000000000004e-05 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637345+0000", 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7735999999999999e-05 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647845+0000", 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010500012 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647910+0000", 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5662999999999998e-05 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12676 3.1b 3:d8a99671:::rbd_data.115262cdab6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 15.399464678999999, 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.074765842999999998, 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:09.109 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12676 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:09.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604445+0000", 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1399999999999998e-06 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638570+0000", 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.034124814000000003 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:09.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638586+0000", 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6460000000000002e-05 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638611+0000", 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5216e-05 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638676+0000", 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5048999999999993e-05 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638792+0000", 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115233 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.640186+0000", 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013940090000000001 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657756+0000", 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017570240000000001 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679079+0000", 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.021323312000000001 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679150+0000", 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1086000000000003e-05 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679209+0000", 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8284e-05 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.114 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:09.115 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:09.115 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:09.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:09.117 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:09.114+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:09.118 INFO:teuthology.orchestra.run.smithi139.stdout:2811 read 0x67c0400 thru 0x67c510d (0x4d0e bytes) 2024-09-17T23:30:09.119 INFO:teuthology.orchestra.run.smithi139.stdout:2812 read 0x7efba00 thru 0x7f04dc3 (0x93c4 bytes) 2024-09-17T23:30:09.119 INFO:teuthology.orchestra.run.smithi139.stdout:2814 punch from 0xbcce00 to 0xbd3800, (0x6a00 bytes) 2024-09-17T23:30:09.119 INFO:teuthology.orchestra.run.smithi139.stdout:2818 write 0x2066400 thru 0x206c607 (0x6208 bytes) 2024-09-17T23:30:09.119 INFO:teuthology.orchestra.run.smithi139.stdout:2820 write 0x8523200 thru 0x852ba09 (0x880a bytes) 2024-09-17T23:30:09.119 INFO:teuthology.orchestra.run.smithi139.stdout:2821 punch from 0x4516c00 to 0x4522c00, (0xc000 bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2823 write 0x3b22e00 thru 0x3b263a7 (0x35a8 bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2826 read 0x8709e00 thru 0x8714e90 (0xb091 bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2828 write 0x96e6600 thru 0x96ed308 (0x6d09 bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2829 punch from 0x8303400 to 0x8306600, (0x3200 bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2830 punch from 0x5993400 to 0x599b200, (0x7e00 bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2832 write 0x1c5f200 thru 0x1c6568d (0x648e bytes) 2024-09-17T23:30:09.120 INFO:teuthology.orchestra.run.smithi139.stdout:2836 punch from 0x64b8600 to 0x64baa00, (0x2400 bytes) 2024-09-17T23:30:09.121 INFO:teuthology.orchestra.run.smithi139.stdout:2841 clone 21 order 23 su 524288 sc 15 2024-09-17T23:30:09.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:09.216 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:09.214+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:09.260 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:09.264 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:09.264 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:09.264 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:09.265 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:09.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:09.277 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:09.277 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:09.293 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:09.310 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:09.317 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:09.316+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:09.327 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:09.328 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:09.344 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:09.418 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:09.416+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:09.420 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:09.420 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:09.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:09.519 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:09.517+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:09.577 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:09.577 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:09.586 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:09.627 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:09.625+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:09.721 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:09.719+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:09.733 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:09.733 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:09.742 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:09.826 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:09.824+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:09.889 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:09.889 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:09.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:09.939 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:09.937+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:10.024 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:10.022+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:10.045 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.046 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:10.084 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:10.084 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:10.085 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T23:30:10.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:30:10.093 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:30:10.125 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:10.123+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:10.186 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.187 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:10.242 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:10.240+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:10.327 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:10.325+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 61, 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:30:10.087962+0000", 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:28:57.456516+0000", 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:29:26.866978+0000", 2024-09-17T23:30:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5804032, 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:30:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:30:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:30:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:30:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:30:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.25, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.25, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.25, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.25, 2024-09-17T23:30:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "61", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 23, 2024-09-17T23:30:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 61, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.25, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.25, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-17T23:30:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.25, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.25, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 50, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:30:10.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:30:10.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:30:10.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 36, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:30:10.441 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:30:10.442 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:10.447 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:10.448 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 36, 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:30:10.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 53, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:30:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.495330810546875, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 59, 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:30:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1870185623", 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1870185623", 2024-09-17T23:30:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1870185623", 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1870185623", 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:30:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-17T23:30:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:30:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:30:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:30:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:30:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:30:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:30:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:30:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:10.472 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:10.474 INFO:tasks.thrashosds.thrasher:j is {'epoch': 61, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:30:10.087962+0000', 'last_up_change': '2024-09-17T23:28:57.456516+0000', 'last_in_change': '2024-09-17T23:29:26.866978+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 0.495330810546875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 59, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1870185623}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1870185623}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1870185623}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1870185623}]}, 'public_addr': '172.21.15.37:6809/1870185623', 'cluster_addr': '172.21.15.37:6811/1870185623', 'heartbeat_back_addr': '172.21.15.37:6815/1870185623', 'heartbeat_front_addr': '172.21.15.37:6813/1870185623', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.17', 'osds': [5, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:30:10.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:30:10.474 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.475 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.475 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:10.471+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:10.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:10.518 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T23:30:10.574 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:10.572+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:10.664 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.664 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.677 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:10.673+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:10.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:10.776 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:10.774+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:10.822 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.822 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:10.876 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:10.875+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:30:10.937 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-09-17T23:30:10.372508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":778366694,"num_objects":745,"num_object_clones":217,"num_object_copies":1490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":745,"num_whiteouts":219,"num_read":164463,"num_read_kb":573207,"num_write":8341,"num_write_kb":256145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":14365043,"num_keys_recovered":40,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7309,"ondisk_log_size":7309,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":255176,"kb_used_data":22288,"kb_used_omap":9,"kb_used_meta":232758,"kb_avail":565975864,"statfs":{"total":579820584960,"available":579559284736,"internally_reserved":0,"allocated":22822912,"data_stored":144313223,"data_compressed":6612900,"data_compressed_allocated":13901824,"data_compressed_original":139998208,"omap_allocated":9566,"internal_metadata":238344866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10018116,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":10752,"num_read_kb":85116,"num_write":770,"num_write_kb":4754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2316890,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000686"},"pg_stats":[{"pgid":"3.1f","version":"58'258","reported_seq":3142,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.083097+0000","last_change":"2024-09-17T23:29:16.849137+0000","last_active":"2024-09-17T23:30:08.083097+0000","last_peered":"2024-09-17T23:30:08.083097+0000","last_clean":"2024-09-17T23:30:08.083097+0000","last_became_active":"2024-09-17T23:29:15.605286+0000","last_became_peered":"2024-09-17T23:29:15.605286+0000","last_unstale":"2024-09-17T23:30:08.083097+0000","last_undegraded":"2024-09-17T23:30:08.083097+0000","last_fullsized":"2024-09-17T23:30:08.083097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:04:24.936636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055557299999999996,"stat_sum":{"num_bytes":30745652,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":3819,"num_read_kb":12513,"num_write":289,"num_write_kb":4048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"55'230","reported_seq":4765,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081969+0000","last_change":"2024-09-17T23:29:27.994821+0000","last_active":"2024-09-17T23:30:08.081969+0000","last_peered":"2024-09-17T23:30:08.081969+0000","last_clean":"2024-09-17T23:30:08.081969+0000","last_became_active":"2024-09-17T23:29:27.894511+0000","last_became_peered":"2024-09-17T23:29:27.894511+0000","last_unstale":"2024-09-17T23:30:08.081969+0000","last_undegraded":"2024-09-17T23:30:08.081969+0000","last_fullsized":"2024-09-17T23:30:08.081969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:05:44.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055521100000000001,"stat_sum":{"num_bytes":80398045,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4927,"num_read_kb":32042,"num_write":306,"num_write_kb":8247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":4447268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"58'126","reported_seq":1868,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085046+0000","last_change":"2024-09-17T23:29:33.928943+0000","last_active":"2024-09-17T23:30:08.085046+0000","last_peered":"2024-09-17T23:30:08.085046+0000","last_clean":"2024-09-17T23:30:08.085046+0000","last_became_active":"2024-09-17T23:29:33.928547+0000","last_became_peered":"2024-09-17T23:29:33.928547+0000","last_unstale":"2024-09-17T23:30:08.085046+0000","last_undegraded":"2024-09-17T23:30:08.085046+0000","last_fullsized":"2024-09-17T23:30:08.085046+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:49.110387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476274,"stat_sum":{"num_bytes":16676352,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":1769,"num_read_kb":12365,"num_write":118,"num_write_kb":5176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":187904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"55'113","reported_seq":6167,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085157+0000","last_change":"2024-09-17T23:29:39.959779+0000","last_active":"2024-09-17T23:30:08.085157+0000","last_peered":"2024-09-17T23:30:08.085157+0000","last_clean":"2024-09-17T23:30:08.085157+0000","last_became_active":"2024-09-17T23:29:39.959404+0000","last_became_peered":"2024-09-17T23:29:39.959404+0000","last_unstale":"2024-09-17T23:30:08.085157+0000","last_undegraded":"2024-09-17T23:30:08.085157+0000","last_fullsized":"2024-09-17T23:30:08.085157+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:51.577335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263599,"stat_sum":{"num_bytes":14323728,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":5977,"num_read_kb":17985,"num_write":101,"num_write_kb":2141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"58'150","reported_seq":15271,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081841+0000","last_change":"2024-09-17T23:29:16.844175+0000","last_active":"2024-09-17T23:30:08.081841+0000","last_peered":"2024-09-17T23:30:08.081841+0000","last_clean":"2024-09-17T23:30:08.081841+0000","last_became_active":"2024-09-17T23:29:15.601606+0000","last_became_peered":"2024-09-17T23:29:15.601606+0000","last_unstale":"2024-09-17T23:30:08.081841+0000","last_undegraded":"2024-09-17T23:30:08.081841+0000","last_fullsized":"2024-09-17T23:30:08.081841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:16.800414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046809400000000001,"stat_sum":{"num_bytes":54744849,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":15125,"num_read_kb":38865,"num_write":159,"num_write_kb":18717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"58'551","reported_seq":1390,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084859+0000","last_change":"2024-09-17T23:29:59.097118+0000","last_active":"2024-09-17T23:30:08.084859+0000","last_peered":"2024-09-17T23:30:08.084859+0000","last_clean":"2024-09-17T23:30:08.084859+0000","last_became_active":"2024-09-17T23:29:59.041866+0000","last_became_peered":"2024-09-17T23:29:59.041866+0000","last_unstale":"2024-09-17T23:30:08.084859+0000","last_undegraded":"2024-09-17T23:30:08.084859+0000","last_fullsized":"2024-09-17T23:30:08.084859+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:30.967604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001041527,"stat_sum":{"num_bytes":22158013,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":5399,"num_read_kb":11072,"num_write":970,"num_write_kb":1599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":179747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"58'204","reported_seq":2242,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084753+0000","last_change":"2024-09-17T23:29:16.855904+0000","last_active":"2024-09-17T23:30:08.084753+0000","last_peered":"2024-09-17T23:30:08.084753+0000","last_clean":"2024-09-17T23:30:08.084753+0000","last_became_active":"2024-09-17T23:29:15.602040+0000","last_became_peered":"2024-09-17T23:29:15.602040+0000","last_unstale":"2024-09-17T23:30:08.084753+0000","last_undegraded":"2024-09-17T23:30:08.084753+0000","last_fullsized":"2024-09-17T23:30:08.084753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:19.915239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001306843,"stat_sum":{"num_bytes":15059999,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":2919,"num_read_kb":5162,"num_write":171,"num_write_kb":1112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"55'238","reported_seq":2381,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081654+0000","last_change":"2024-09-17T23:29:16.844092+0000","last_active":"2024-09-17T23:30:08.081654+0000","last_peered":"2024-09-17T23:30:08.081654+0000","last_clean":"2024-09-17T23:30:08.081654+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:30:08.081654+0000","last_undegraded":"2024-09-17T23:30:08.081654+0000","last_fullsized":"2024-09-17T23:30:08.081654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:02.357215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045869699999999999,"stat_sum":{"num_bytes":17261108,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":2972,"num_read_kb":14776,"num_write":273,"num_write_kb":17319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"58'315","reported_seq":4857,"reported_epoch":58,"state":"peering","last_fresh":"2024-09-17T23:30:08.078901+0000","last_change":"2024-09-17T23:30:08.083471+0000","last_active":"2024-09-17T23:30:08.083471+0000","last_peered":"2024-09-17T23:30:08.078901+0000","last_clean":"2024-09-17T23:30:05.066761+0000","last_became_active":"2024-09-17T23:30:07.144051+0000","last_became_peered":"2024-09-17T23:30:07.144051+0000","last_unstale":"2024-09-17T23:30:08.078901+0000","last_undegraded":"2024-09-17T23:30:08.078901+0000","last_fullsized":"2024-09-17T23:30:06.850339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050274800000000002,"stat_sum":{"num_bytes":35807322,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":5665,"num_read_kb":9987,"num_write":322,"num_write_kb":7756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2316890,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"54'137","reported_seq":1974,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081848+0000","last_change":"2024-09-17T23:29:33.926843+0000","last_active":"2024-09-17T23:30:08.081848+0000","last_peered":"2024-09-17T23:30:08.081848+0000","last_clean":"2024-09-17T23:30:08.081848+0000","last_became_active":"2024-09-17T23:29:33.926463+0000","last_became_peered":"2024-09-17T23:29:33.926463+0000","last_unstale":"2024-09-17T23:30:08.081848+0000","last_undegraded":"2024-09-17T23:30:08.081848+0000","last_fullsized":"2024-09-17T23:30:08.081848+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:44:42.138238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099808899999999996,"stat_sum":{"num_bytes":14573104,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":2529,"num_read_kb":14471,"num_write":137,"num_write_kb":3846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":42544,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"58'273","reported_seq":10250,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085221+0000","last_change":"2024-09-17T23:29:16.851584+0000","last_active":"2024-09-17T23:30:08.085221+0000","last_peered":"2024-09-17T23:30:08.085221+0000","last_clean":"2024-09-17T23:30:08.085221+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:30:08.085221+0000","last_undegraded":"2024-09-17T23:30:08.085221+0000","last_fullsized":"2024-09-17T23:30:08.085221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:55:01.639748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073596200000000005,"stat_sum":{"num_bytes":27632680,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":3,"num_read":11149,"num_read_kb":16585,"num_write":277,"num_write_kb":3141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"55'390","reported_seq":5551,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.089451+0000","last_change":"2024-09-17T23:29:28.004020+0000","last_active":"2024-09-17T23:30:08.089451+0000","last_peered":"2024-09-17T23:30:08.089451+0000","last_clean":"2024-09-17T23:30:08.089451+0000","last_became_active":"2024-09-17T23:29:27.894409+0000","last_became_peered":"2024-09-17T23:29:27.894409+0000","last_unstale":"2024-09-17T23:30:08.089451+0000","last_undegraded":"2024-09-17T23:30:08.089451+0000","last_fullsized":"2024-09-17T23:30:08.089451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:12:47.721647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013571150000000001,"stat_sum":{"num_bytes":12967009,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":6599,"num_read_kb":20279,"num_write":344,"num_write_kb":17215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":42496,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085452+0000","last_change":"2024-09-17T23:29:39.959940+0000","last_active":"2024-09-17T23:30:08.085452+0000","last_peered":"2024-09-17T23:30:08.085452+0000","last_clean":"2024-09-17T23:30:08.085452+0000","last_became_active":"2024-09-17T23:29:39.959465+0000","last_became_peered":"2024-09-17T23:29:39.959465+0000","last_unstale":"2024-09-17T23:30:08.085452+0000","last_undegraded":"2024-09-17T23:30:08.085452+0000","last_fullsized":"2024-09-17T23:30:08.085452+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:35.867019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"58'437","reported_seq":1209,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084821+0000","last_change":"2024-09-17T23:29:16.856135+0000","last_active":"2024-09-17T23:30:08.084821+0000","last_peered":"2024-09-17T23:30:08.084821+0000","last_clean":"2024-09-17T23:30:08.084821+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:30:08.084821+0000","last_undegraded":"2024-09-17T23:30:08.084821+0000","last_fullsized":"2024-09-17T23:30:08.084821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:38.296889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001532662,"stat_sum":{"num_bytes":5517911,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2587,"num_read_kb":7881,"num_write":727,"num_write_kb":5625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084959+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:30:08.084959+0000","last_peered":"2024-09-17T23:30:08.084959+0000","last_clean":"2024-09-17T23:30:08.084959+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:30:08.084959+0000","last_undegraded":"2024-09-17T23:30:08.084959+0000","last_fullsized":"2024-09-17T23:30:08.084959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"58'185","reported_seq":7058,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085160+0000","last_change":"2024-09-17T23:29:16.855828+0000","last_active":"2024-09-17T23:30:08.085160+0000","last_peered":"2024-09-17T23:30:08.085160+0000","last_clean":"2024-09-17T23:30:08.085160+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:30:08.085160+0000","last_undegraded":"2024-09-17T23:30:08.085160+0000","last_fullsized":"2024-09-17T23:30:08.085160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:27:55.545805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012515670000000001,"stat_sum":{"num_bytes":15504427,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":7837,"num_read_kb":22290,"num_write":136,"num_write_kb":19662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084874+0000","last_change":"2024-09-17T23:29:39.958959+0000","last_active":"2024-09-17T23:30:08.084874+0000","last_peered":"2024-09-17T23:30:08.084874+0000","last_clean":"2024-09-17T23:30:08.084874+0000","last_became_active":"2024-09-17T23:29:39.958620+0000","last_became_peered":"2024-09-17T23:29:39.958620+0000","last_unstale":"2024-09-17T23:30:08.084874+0000","last_undegraded":"2024-09-17T23:30:08.084874+0000","last_fullsized":"2024-09-17T23:30:08.084874+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:19.215313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"58'211","reported_seq":5125,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084821+0000","last_change":"2024-09-17T23:30:01.851090+0000","last_active":"2024-09-17T23:30:08.084821+0000","last_peered":"2024-09-17T23:30:08.084821+0000","last_clean":"2024-09-17T23:30:08.084821+0000","last_became_active":"2024-09-17T23:30:01.850019+0000","last_became_peered":"2024-09-17T23:30:01.850019+0000","last_unstale":"2024-09-17T23:30:08.084821+0000","last_undegraded":"2024-09-17T23:30:08.084821+0000","last_fullsized":"2024-09-17T23:30:08.084821+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:24:38.544104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010554410000000001,"stat_sum":{"num_bytes":9613342,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":3227,"num_read_kb":8233,"num_write":270,"num_write_kb":3465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2524190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.083084+0000","last_change":"2024-09-17T23:29:08.767780+0000","last_active":"2024-09-17T23:30:08.083084+0000","last_peered":"2024-09-17T23:30:08.083084+0000","last_clean":"2024-09-17T23:30:08.083084+0000","last_became_active":"2024-09-17T23:29:05.499286+0000","last_became_peered":"2024-09-17T23:29:05.499286+0000","last_unstale":"2024-09-17T23:30:08.083084+0000","last_undegraded":"2024-09-17T23:30:08.083084+0000","last_fullsized":"2024-09-17T23:30:08.083084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:47:22.262418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"58'103","reported_seq":7956,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081781+0000","last_change":"2024-09-17T23:29:33.928884+0000","last_active":"2024-09-17T23:30:08.081781+0000","last_peered":"2024-09-17T23:30:08.081781+0000","last_clean":"2024-09-17T23:30:08.081781+0000","last_became_active":"2024-09-17T23:29:33.928630+0000","last_became_peered":"2024-09-17T23:29:33.928630+0000","last_unstale":"2024-09-17T23:30:08.081781+0000","last_undegraded":"2024-09-17T23:30:08.081781+0000","last_fullsized":"2024-09-17T23:30:08.081781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:49.918628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17972768,"num_objects":30,"num_object_clones":10,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":7814,"num_read_kb":16448,"num_write":106,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"58'129","reported_seq":2847,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085034+0000","last_change":"2024-09-17T23:29:27.969780+0000","last_active":"2024-09-17T23:30:08.085034+0000","last_peered":"2024-09-17T23:30:08.085034+0000","last_clean":"2024-09-17T23:30:08.085034+0000","last_became_active":"2024-09-17T23:29:27.896108+0000","last_became_peered":"2024-09-17T23:29:27.896108+0000","last_unstale":"2024-09-17T23:30:08.085034+0000","last_undegraded":"2024-09-17T23:30:08.085034+0000","last_fullsized":"2024-09-17T23:30:08.085034+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:36.620765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000582823,"stat_sum":{"num_bytes":17168896,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":3096,"num_read_kb":10307,"num_write":105,"num_write_kb":2577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":219648,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085549+0000","last_change":"2024-09-17T23:29:07.502518+0000","last_active":"2024-09-17T23:30:08.085549+0000","last_peered":"2024-09-17T23:30:08.085549+0000","last_clean":"2024-09-17T23:30:08.085549+0000","last_became_active":"2024-09-17T23:29:05.499700+0000","last_became_peered":"2024-09-17T23:29:05.499700+0000","last_unstale":"2024-09-17T23:30:08.085549+0000","last_undegraded":"2024-09-17T23:30:08.085549+0000","last_fullsized":"2024-09-17T23:30:08.085549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:56.188652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"58'205","reported_seq":4498,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.083062+0000","last_change":"2024-09-17T23:29:27.926562+0000","last_active":"2024-09-17T23:30:08.083062+0000","last_peered":"2024-09-17T23:30:08.083062+0000","last_clean":"2024-09-17T23:30:08.083062+0000","last_became_active":"2024-09-17T23:29:27.893418+0000","last_became_peered":"2024-09-17T23:29:27.893418+0000","last_unstale":"2024-09-17T23:30:08.083062+0000","last_undegraded":"2024-09-17T23:30:08.083062+0000","last_fullsized":"2024-09-17T23:30:08.083062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:09:13.529557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363521,"stat_sum":{"num_bytes":10143843,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":5460,"num_read_kb":14480,"num_write":217,"num_write_kb":986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":99,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081746+0000","last_change":"2024-09-17T23:29:09.377458+0000","last_active":"2024-09-17T23:30:08.081746+0000","last_peered":"2024-09-17T23:30:08.081746+0000","last_clean":"2024-09-17T23:30:08.081746+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:30:08.081746+0000","last_undegraded":"2024-09-17T23:30:08.081746+0000","last_fullsized":"2024-09-17T23:30:08.081746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:59.106784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"58'315","reported_seq":3340,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084766+0000","last_change":"2024-09-17T23:29:30.917501+0000","last_active":"2024-09-17T23:30:08.084766+0000","last_peered":"2024-09-17T23:30:08.084766+0000","last_clean":"2024-09-17T23:30:08.084766+0000","last_became_active":"2024-09-17T23:29:30.917041+0000","last_became_peered":"2024-09-17T23:29:30.917041+0000","last_unstale":"2024-09-17T23:30:08.084766+0000","last_undegraded":"2024-09-17T23:30:08.084766+0000","last_fullsized":"2024-09-17T23:30:08.084766+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:33:12.911292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00105678,"stat_sum":{"num_bytes":26297531,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":4810,"num_read_kb":6172,"num_write":220,"num_write_kb":1658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":86528,"num_keys_recovered":17,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084706+0000","last_change":"2024-09-17T23:29:59.042329+0000","last_active":"2024-09-17T23:30:08.084706+0000","last_peered":"2024-09-17T23:30:08.084706+0000","last_clean":"2024-09-17T23:30:08.084706+0000","last_became_active":"2024-09-17T23:29:59.042013+0000","last_became_peered":"2024-09-17T23:29:59.042013+0000","last_unstale":"2024-09-17T23:30:08.084706+0000","last_undegraded":"2024-09-17T23:30:08.084706+0000","last_fullsized":"2024-09-17T23:30:08.084706+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:18.529822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"58'198","reported_seq":1976,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081721+0000","last_change":"2024-09-17T23:29:27.947652+0000","last_active":"2024-09-17T23:30:08.081721+0000","last_peered":"2024-09-17T23:30:08.081721+0000","last_clean":"2024-09-17T23:30:08.081721+0000","last_became_active":"2024-09-17T23:29:27.894483+0000","last_became_peered":"2024-09-17T23:29:27.894483+0000","last_unstale":"2024-09-17T23:30:08.081721+0000","last_undegraded":"2024-09-17T23:30:08.081721+0000","last_fullsized":"2024-09-17T23:30:08.081721+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:36.351563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047998399999999998,"stat_sum":{"num_bytes":23587937,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":2025,"num_read_kb":21882,"num_write":240,"num_write_kb":19714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":97,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.083018+0000","last_change":"2024-09-17T23:29:39.959552+0000","last_active":"2024-09-17T23:30:08.083018+0000","last_peered":"2024-09-17T23:30:08.083018+0000","last_clean":"2024-09-17T23:30:08.083018+0000","last_became_active":"2024-09-17T23:29:39.959224+0000","last_became_peered":"2024-09-17T23:29:39.959224+0000","last_unstale":"2024-09-17T23:30:08.083018+0000","last_undegraded":"2024-09-17T23:30:08.083018+0000","last_fullsized":"2024-09-17T23:30:08.083018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:35.806577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":206,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.083072+0000","last_change":"2024-09-17T23:29:39.959742+0000","last_active":"2024-09-17T23:30:08.083072+0000","last_peered":"2024-09-17T23:30:08.083072+0000","last_clean":"2024-09-17T23:30:08.083072+0000","last_became_active":"2024-09-17T23:29:39.959373+0000","last_became_peered":"2024-09-17T23:29:39.959373+0000","last_unstale":"2024-09-17T23:30:08.083072+0000","last_undegraded":"2024-09-17T23:30:08.083072+0000","last_fullsized":"2024-09-17T23:30:08.083072+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:43:46.897381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"58'220","reported_seq":9029,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084807+0000","last_change":"2024-09-17T23:29:27.961774+0000","last_active":"2024-09-17T23:30:08.084807+0000","last_peered":"2024-09-17T23:30:08.084807+0000","last_clean":"2024-09-17T23:30:08.084807+0000","last_became_active":"2024-09-17T23:29:27.893544+0000","last_became_peered":"2024-09-17T23:29:27.893544+0000","last_unstale":"2024-09-17T23:30:08.084807+0000","last_undegraded":"2024-09-17T23:30:08.084807+0000","last_fullsized":"2024-09-17T23:30:08.084807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:47:11.432029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001536372,"stat_sum":{"num_bytes":22973693,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":9813,"num_read_kb":26982,"num_write":192,"num_write_kb":3270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":46096,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"54'172","reported_seq":4076,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084865+0000","last_change":"2024-09-17T23:29:16.850836+0000","last_active":"2024-09-17T23:30:08.084865+0000","last_peered":"2024-09-17T23:30:08.084865+0000","last_clean":"2024-09-17T23:30:08.084865+0000","last_became_active":"2024-09-17T23:29:15.602469+0000","last_became_peered":"2024-09-17T23:29:15.602469+0000","last_unstale":"2024-09-17T23:30:08.084865+0000","last_undegraded":"2024-09-17T23:30:08.084865+0000","last_fullsized":"2024-09-17T23:30:08.084865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:32:39.446582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483418,"stat_sum":{"num_bytes":12578853,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":2876,"num_read_kb":20256,"num_write":147,"num_write_kb":18655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"58'247","reported_seq":4803,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084996+0000","last_change":"2024-09-17T23:29:16.851336+0000","last_active":"2024-09-17T23:30:08.084996+0000","last_peered":"2024-09-17T23:30:08.084996+0000","last_clean":"2024-09-17T23:30:08.084996+0000","last_became_active":"2024-09-17T23:29:15.605424+0000","last_became_peered":"2024-09-17T23:29:15.605424+0000","last_unstale":"2024-09-17T23:30:08.084996+0000","last_undegraded":"2024-09-17T23:30:08.084996+0000","last_fullsized":"2024-09-17T23:30:08.084996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:38:13.348028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060532200000000004,"stat_sum":{"num_bytes":12512784,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":5523,"num_read_kb":23461,"num_write":197,"num_write_kb":3863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"58'130","reported_seq":2292,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.081734+0000","last_change":"2024-09-17T23:29:16.844405+0000","last_active":"2024-09-17T23:30:08.081734+0000","last_peered":"2024-09-17T23:30:08.081734+0000","last_clean":"2024-09-17T23:30:08.081734+0000","last_became_active":"2024-09-17T23:29:15.601213+0000","last_became_peered":"2024-09-17T23:29:15.601213+0000","last_unstale":"2024-09-17T23:30:08.081734+0000","last_undegraded":"2024-09-17T23:30:08.081734+0000","last_fullsized":"2024-09-17T23:30:08.081734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:15:59.271582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049268800000000002,"stat_sum":{"num_bytes":23279648,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2613,"num_read_kb":7483,"num_write":111,"num_write_kb":6048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"55'263","reported_seq":2718,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085180+0000","last_change":"2024-09-17T23:29:16.851583+0000","last_active":"2024-09-17T23:30:08.085180+0000","last_peered":"2024-09-17T23:30:08.085180+0000","last_clean":"2024-09-17T23:30:08.085180+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:30:08.085180+0000","last_undegraded":"2024-09-17T23:30:08.085180+0000","last_fullsized":"2024-09-17T23:30:08.085180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:25.827599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187900000000002,"stat_sum":{"num_bytes":9347775,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":3732,"num_read_kb":16923,"num_write":287,"num_write_kb":3986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"58'117","reported_seq":6535,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.089416+0000","last_change":"2024-09-17T23:29:16.847782+0000","last_active":"2024-09-17T23:30:08.089416+0000","last_peered":"2024-09-17T23:30:08.089416+0000","last_clean":"2024-09-17T23:30:08.089416+0000","last_became_active":"2024-09-17T23:29:15.599701+0000","last_became_peered":"2024-09-17T23:29:15.599701+0000","last_unstale":"2024-09-17T23:30:08.089416+0000","last_undegraded":"2024-09-17T23:30:08.089416+0000","last_fullsized":"2024-09-17T23:30:08.089416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:41.889333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095122,"stat_sum":{"num_bytes":26479684,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":6410,"num_read_kb":18699,"num_write":133,"num_write_kb":8415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"59'342","reported_seq":4562,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.087328+0000","last_change":"2024-09-17T23:29:33.928677+0000","last_active":"2024-09-17T23:30:08.087328+0000","last_peered":"2024-09-17T23:30:08.087328+0000","last_clean":"2024-09-17T23:30:08.087328+0000","last_became_active":"2024-09-17T23:29:33.928254+0000","last_became_peered":"2024-09-17T23:29:33.928254+0000","last_unstale":"2024-09-17T23:30:08.087328+0000","last_undegraded":"2024-09-17T23:30:08.087328+0000","last_fullsized":"2024-09-17T23:30:08.087328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:47.735382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093123200000000004,"stat_sum":{"num_bytes":14391004,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":6200,"num_read_kb":16679,"num_write":415,"num_write_kb":3233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"58'103","reported_seq":2206,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085438+0000","last_change":"2024-09-17T23:29:16.851833+0000","last_active":"2024-09-17T23:30:08.085438+0000","last_peered":"2024-09-17T23:30:08.085438+0000","last_clean":"2024-09-17T23:30:08.085438+0000","last_became_active":"2024-09-17T23:29:15.606245+0000","last_became_peered":"2024-09-17T23:29:15.606245+0000","last_unstale":"2024-09-17T23:30:08.085438+0000","last_undegraded":"2024-09-17T23:30:08.085438+0000","last_fullsized":"2024-09-17T23:30:08.085438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:06.348860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078560200000000005,"stat_sum":{"num_bytes":6846004,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2103,"num_read_kb":14906,"num_write":125,"num_write_kb":2764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"59'211","reported_seq":1255,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084936+0000","last_change":"2024-09-17T23:29:59.068541+0000","last_active":"2024-09-17T23:30:08.084936+0000","last_peered":"2024-09-17T23:30:08.084936+0000","last_clean":"2024-09-17T23:30:08.084936+0000","last_became_active":"2024-09-17T23:29:59.037097+0000","last_became_peered":"2024-09-17T23:29:59.037097+0000","last_unstale":"2024-09-17T23:30:08.084936+0000","last_undegraded":"2024-09-17T23:30:08.084936+0000","last_fullsized":"2024-09-17T23:30:08.084936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:15:45.271979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056375800000000003,"stat_sum":{"num_bytes":21926993,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1895,"num_read_kb":14875,"num_write":282,"num_write_kb":18410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":573478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"58'136","reported_seq":3404,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.083010+0000","last_change":"2024-09-17T23:29:39.959636+0000","last_active":"2024-09-17T23:30:08.083010+0000","last_peered":"2024-09-17T23:30:08.083010+0000","last_clean":"2024-09-17T23:30:08.083010+0000","last_became_active":"2024-09-17T23:29:39.959197+0000","last_became_peered":"2024-09-17T23:29:39.959197+0000","last_unstale":"2024-09-17T23:30:08.083010+0000","last_undegraded":"2024-09-17T23:30:08.083010+0000","last_fullsized":"2024-09-17T23:30:08.083010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:45:41.865218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043017799999999997,"stat_sum":{"num_bytes":50975744,"num_objects":28,"num_object_clones":11,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":4037,"num_read_kb":24789,"num_write":127,"num_write_kb":3679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":110592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"58'128","reported_seq":2677,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.084880+0000","last_change":"2024-09-17T23:29:59.135036+0000","last_active":"2024-09-17T23:30:08.084880+0000","last_peered":"2024-09-17T23:30:08.084880+0000","last_clean":"2024-09-17T23:30:08.084880+0000","last_became_active":"2024-09-17T23:29:59.042434+0000","last_became_peered":"2024-09-17T23:29:59.042434+0000","last_unstale":"2024-09-17T23:30:08.084880+0000","last_undegraded":"2024-09-17T23:30:08.084880+0000","last_fullsized":"2024-09-17T23:30:08.084880+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:42.924117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034516700000000001,"stat_sum":{"num_bytes":62203904,"num_objects":23,"num_object_clones":9,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":2420,"num_read_kb":15439,"num_write":146,"num_write_kb":6177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":3587584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"58'352","reported_seq":10393,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T23:30:08.085017+0000","last_change":"2024-09-17T23:29:39.959921+0000","last_active":"2024-09-17T23:30:08.085017+0000","last_peered":"2024-09-17T23:30:08.085017+0000","last_clean":"2024-09-17T23:30:08.085017+0000","last_became_active":"2024-09-17T23:29:39.959472+0000","last_became_peered":"2024-09-17T23:29:39.959472+0000","last_unstale":"2024-09-17T23:30:08.085017+0000","last_undegraded":"2024-09-17T23:30:08.085017+0000","last_fullsized":"2024-09-17T23:30:08.085017+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:34:30.968129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001102308,"stat_sum":{"num_bytes":46105705,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":11020,"num_read_kb":58820,"num_write":530,"num_write_kb":28246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":777776307,"num_objects":741,"num_object_clones":217,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":741,"num_whiteouts":219,"num_read":164347,"num_read_kb":573107,"num_write":8180,"num_write_kb":253727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":14365043,"num_keys_recovered":40,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18857984,"data_stored":141809600,"data_compressed":6590247,"data_compressed_allocated":13840384,"data_compressed_original":139039744,"omap_allocated":0,"internal_metadata":0},"log_size":7189,"ondisk_log_size":7189,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640273,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36844,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334996,"statfs":{"total":96636764160,"available":96599035904,"internally_reserved":0,"allocated":1523712,"data_stored":6838546,"data_compressed":278155,"data_compressed_allocated":647168,"data_compressed_original":6452736,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640273,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42236,"kb_used_data":2724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329604,"statfs":{"total":96636764160,"available":96593514496,"internally_reserved":0,"allocated":2789376,"data_stored":16545621,"data_compressed":715967,"data_compressed_allocated":1454080,"data_compressed_original":15954944,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640273,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44264,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327576,"statfs":{"total":96636764160,"available":96591437824,"internally_reserved":0,"allocated":4874240,"data_stored":32893601,"data_compressed":1396357,"data_compressed_allocated":3117056,"data_compressed_original":32045056,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640273,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43972,"kb_used_data":4468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327868,"statfs":{"total":96636764160,"available":96591736832,"internally_reserved":0,"allocated":4575232,"data_stored":26773269,"data_compressed":1446649,"data_compressed_allocated":2822144,"data_compressed_original":25918976,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640274,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42988,"kb_used_data":3480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328852,"statfs":{"total":96636764160,"available":96592744448,"internally_reserved":0,"allocated":3563520,"data_stored":24011461,"data_compressed":1019370,"data_compressed_allocated":2170880,"data_compressed_original":23305216,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640273,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44872,"kb_used_data":5368,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326968,"statfs":{"total":96636764160,"available":96590815232,"internally_reserved":0,"allocated":5496832,"data_stored":37250725,"data_compressed":1756402,"data_compressed_allocated":3690496,"data_compressed_original":36321280,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4907008,"data_stored":37087941,"data_compressed":1756473,"data_compressed_allocated":3698688,"data_compressed_original":36382720,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2859008,"data_stored":23169700,"data_compressed":1014428,"data_compressed_allocated":2134016,"data_compressed_original":22731776,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3928064,"data_stored":26516740,"data_compressed":1426223,"data_compressed_allocated":2805760,"data_compressed_original":25910784,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4247552,"data_stored":32764064,"data_compressed":1396464,"data_compressed_allocated":3129344,"data_compressed_original":32167936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2109440,"data_stored":16276930,"data_compressed":715967,"data_compressed_allocated":1454080,"data_compressed_original":15954944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":806912,"data_stored":5994225,"data_compressed":280692,"data_compressed_allocated":618496,"data_compressed_original":5891584,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:30:10.939 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [3, 2, 5, 0] 2024-09-17T23:30:10.939 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1', '3', '2', '5', '0'] 2024-09-17T23:30:10.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1 3 2 5 0 2024-09-17T23:30:10.983 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:10.982+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:10.989 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:10.990 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:10.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:11.028 INFO:teuthology.orchestra.run.smithi139.stdout:2842 read 0x4159a00 thru 0x415bc2c (0x222d bytes) 2024-09-17T23:30:11.029 INFO:teuthology.orchestra.run.smithi139.stdout:2843 write 0x1f4e00 thru 0x1fa861 (0x5a62 bytes) 2024-09-17T23:30:11.032 INFO:teuthology.orchestra.run.smithi139.stdout:2845 read 0x8ada000 thru 0x8ae068b (0x668c bytes) 2024-09-17T23:30:11.032 INFO:teuthology.orchestra.run.smithi139.stdout:2846 write 0x4aad600 thru 0x4ab3842 (0x6243 bytes) 2024-09-17T23:30:11.033 INFO:teuthology.orchestra.run.smithi139.stdout:2847 read 0x8544400 thru 0x854f613 (0xb214 bytes) 2024-09-17T23:30:11.033 INFO:teuthology.orchestra.run.smithi139.stdout:2848 write 0x20c6e00 thru 0x20c92e7 (0x24e8 bytes) 2024-09-17T23:30:11.035 INFO:teuthology.orchestra.run.smithi139.stdout:2849 punch from 0xa1b3800 to 0xa1b6800, (0x3000 bytes) 2024-09-17T23:30:11.050 INFO:teuthology.orchestra.run.smithi139.stdout:2851 write 0x2199e00 thru 0x21a6ef5 (0xd0f6 bytes) 2024-09-17T23:30:11.070 INFO:teuthology.orchestra.run.smithi139.stdout:2852 write 0xe672e00 thru 0xe6789b8 (0x5bb9 bytes) 2024-09-17T23:30:11.071 INFO:teuthology.orchestra.run.smithi139.stdout:2855 write 0x4103400 thru 0x410981f (0x6420 bytes) 2024-09-17T23:30:11.071 INFO:teuthology.orchestra.run.smithi139.stdout:2856 read 0xe17a200 thru 0xe184b25 (0xa926 bytes) 2024-09-17T23:30:11.080 INFO:teuthology.orchestra.run.smithi139.stdout:2857 trunc from 0xe678a00 to 0x7014c00 2024-09-17T23:30:11.083 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:11.082+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:11.129 INFO:teuthology.orchestra.run.smithi139.stdout:2858 write 0x7437e00 thru 0x7441afd (0x9cfe bytes) 2024-09-17T23:30:11.133 INFO:teuthology.orchestra.run.smithi139.stdout:2859 punch from 0x6d31200 to 0x6d32400, (0x1200 bytes) 2024-09-17T23:30:11.133 INFO:teuthology.orchestra.run.smithi139.stdout:2860 punch from 0xaaf000 to 0xab4000, (0x5000 bytes) 2024-09-17T23:30:11.136 INFO:teuthology.orchestra.run.smithi139.stdout:2863 trunc from 0x7441c00 to 0x552d200 2024-09-17T23:30:11.149 INFO:teuthology.orchestra.run.smithi139.stdout:2864 read 0x1a5200 thru 0x1b1df6 (0xcbf7 bytes) 2024-09-17T23:30:11.154 INFO:teuthology.orchestra.run.smithi139.stdout:2865 read 0x2f06400 thru 0x2f0ba9e (0x569f bytes) 2024-09-17T23:30:11.158 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:11.158 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:11.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:11.169 INFO:teuthology.orchestra.run.smithi139.stdout:2866 write 0xa0c6600 thru 0xa0cc698 (0x6099 bytes) 2024-09-17T23:30:11.178 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:11.176+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:11.191 INFO:teuthology.orchestra.run.smithi139.stdout:2870 write 0xc938400 thru 0xc9421cb (0x9dcc bytes) 2024-09-17T23:30:11.193 INFO:teuthology.orchestra.run.smithi139.stdout:2875 read 0x44c9600 thru 0x44cee83 (0x5884 bytes) 2024-09-17T23:30:11.193 INFO:teuthology.orchestra.run.smithi139.stdout:2876 write 0xa166000 thru 0xa171986 (0xb987 bytes) 2024-09-17T23:30:11.193 INFO:teuthology.orchestra.run.smithi139.stdout:2879 write 0x23f5200 thru 0x24000b6 (0xaeb7 bytes) 2024-09-17T23:30:11.194 INFO:teuthology.orchestra.run.smithi139.stdout:2880 read 0x1fc6e00 thru 0x1fd69fc (0xfbfd bytes) 2024-09-17T23:30:11.194 INFO:teuthology.orchestra.run.smithi139.stdout:2882 write 0x2dda00 thru 0x2ed77b (0xfd7c bytes) 2024-09-17T23:30:11.195 INFO:teuthology.orchestra.run.smithi139.stdout:2884 read 0x39e7600 thru 0x39eda11 (0x6412 bytes) 2024-09-17T23:30:11.195 INFO:teuthology.orchestra.run.smithi139.stdout:2885 write 0x76de400 thru 0x76e336e (0x4f6f bytes) 2024-09-17T23:30:11.196 INFO:teuthology.orchestra.run.smithi139.stdout:2886 write 0x538b200 thru 0x5393452 (0x8253 bytes) 2024-09-17T23:30:11.196 INFO:teuthology.orchestra.run.smithi139.stdout:2887 read 0xa09f600 thru 0xa0aa5c1 (0xafc2 bytes) 2024-09-17T23:30:11.197 INFO:teuthology.orchestra.run.smithi139.stdout:2888 write 0x37afe00 thru 0x37ba095 (0xa296 bytes) 2024-09-17T23:30:11.197 INFO:teuthology.orchestra.run.smithi139.stdout:2890 write 0x29ada00 thru 0x29afc22 (0x2223 bytes) 2024-09-17T23:30:11.198 INFO:teuthology.orchestra.run.smithi139.stdout:2893 read 0x6273a00 thru 0x628296f (0xef70 bytes) 2024-09-17T23:30:11.199 INFO:teuthology.orchestra.run.smithi139.stdout:2895 trunc from 0xc942200 to 0xd731000 2024-09-17T23:30:11.220 INFO:teuthology.orchestra.run.smithi139.stdout:2896 write 0xb48e00 thru 0xb49921 (0xb22 bytes) 2024-09-17T23:30:11.221 INFO:teuthology.orchestra.run.smithi139.stdout:2899 read 0xd308400 thru 0xd30d153 (0x4d54 bytes) 2024-09-17T23:30:11.233 INFO:teuthology.orchestra.run.smithi139.stdout:2900 trunc from 0xd731000 to 0x33a6600 2024-09-17T23:30:11.260 INFO:teuthology.orchestra.run.smithi139.stdout:2908 trunc from 0x33a6600 to 0x5c60800 2024-09-17T23:30:11.275 INFO:teuthology.orchestra.run.smithi139.stdout:2909 write 0x75b800 thru 0x75d601 (0x1e02 bytes) 2024-09-17T23:30:11.279 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:11.278+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:11.297 INFO:teuthology.orchestra.run.smithi139.stdout:2910 write 0xbe31c00 thru 0xbe3385b (0x1c5c bytes) 2024-09-17T23:30:11.298 INFO:teuthology.orchestra.run.smithi139.stdout:2912 punch from 0x246a600 to 0x2472800, (0x8200 bytes) 2024-09-17T23:30:11.299 INFO:teuthology.orchestra.run.smithi139.stdout:2913 write 0x7da2400 thru 0x7dacc6d (0xa86e bytes) 2024-09-17T23:30:11.299 INFO:teuthology.orchestra.run.smithi139.stdout:2914 write 0x72c2000 thru 0x72d0dcd (0xedce bytes) 2024-09-17T23:30:11.300 INFO:teuthology.orchestra.run.smithi139.stdout:2916 punch from 0xb808600 to 0xb810400, (0x7e00 bytes) 2024-09-17T23:30:11.304 INFO:teuthology.orchestra.run.smithi139.stdout:2919 trunc from 0xbe33a00 to 0x8948600 2024-09-17T23:30:11.314 INFO:teuthology.orchestra.run.smithi139.stdout:2920 write 0x587e800 thru 0x588355f (0x4d60 bytes) 2024-09-17T23:30:11.315 INFO:teuthology.orchestra.run.smithi139.stdout:2921 punch from 0x8a0e00 to 0x8a3800, (0x2a00 bytes) 2024-09-17T23:30:11.315 INFO:teuthology.orchestra.run.smithi139.stdout:2923 write 0x7807c00 thru 0x78179f8 (0xfdf9 bytes) 2024-09-17T23:30:11.315 INFO:teuthology.orchestra.run.smithi139.stdout:2924 read 0x635f800 thru 0x6366608 (0x6e09 bytes) 2024-09-17T23:30:11.316 INFO:teuthology.orchestra.run.smithi139.stdout:2925 punch from 0x3d42800 to 0x3d4a600, (0x7e00 bytes) 2024-09-17T23:30:11.316 INFO:teuthology.orchestra.run.smithi139.stdout:2926 read 0xe09000 thru 0xe0bcd4 (0x2cd5 bytes) 2024-09-17T23:30:11.317 INFO:teuthology.orchestra.run.smithi139.stdout:2928 write 0x2e13a00 thru 0x2e2133a (0xd93b bytes) 2024-09-17T23:30:11.318 INFO:teuthology.orchestra.run.smithi139.stdout:2930 trunc from 0x8948600 to 0x8e3e600 2024-09-17T23:30:11.322 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:11.322 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:11.329 INFO:teuthology.orchestra.run.smithi139.stdout:2935 read 0x305ac00 thru 0x3067a1c (0xce1d bytes) 2024-09-17T23:30:11.330 INFO:teuthology.orchestra.run.smithi139.stdout:2937 read 0xf43800 thru 0xf4cc59 (0x945a bytes) 2024-09-17T23:30:11.331 INFO:teuthology.orchestra.run.smithi139.stdout:2938 read 0x87cc00 thru 0x883e10 (0x7211 bytes) 2024-09-17T23:30:11.331 INFO:teuthology.orchestra.run.smithi139.stdout:2939 read 0x5936200 thru 0x594463c (0xe43d bytes) 2024-09-17T23:30:11.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:11.333 INFO:teuthology.orchestra.run.smithi139.stdout:2941 read 0x4a92400 thru 0x4a990fe (0x6cff bytes) 2024-09-17T23:30:11.333 INFO:teuthology.orchestra.run.smithi139.stdout:2943 read 0x2bd2600 thru 0x2bd8816 (0x6217 bytes) 2024-09-17T23:30:11.333 INFO:teuthology.orchestra.run.smithi139.stdout:2946 write 0x50d7a00 thru 0x50ddb3d (0x613e bytes) 2024-09-17T23:30:11.348 INFO:teuthology.orchestra.run.smithi139.stdout:2947 write 0xe0bc800 thru 0xe0ca85b (0xe05c bytes) 2024-09-17T23:30:11.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:11.361 INFO:teuthology.orchestra.run.smithi139.stdout:2951 trunc from 0xe0caa00 to 0x9529200 2024-09-17T23:30:11.375 INFO:teuthology.orchestra.run.smithi139.stdout:2954 read 0x628ca00 thru 0x629bf8a (0xf58b bytes) 2024-09-17T23:30:11.377 INFO:teuthology.orchestra.run.smithi139.stdout:2957 read 0x6826000 thru 0x682bc6d (0x5c6e bytes) 2024-09-17T23:30:11.377 INFO:teuthology.orchestra.run.smithi139.stdout:2958 write 0x90c9a00 thru 0x90d32b6 (0x98b7 bytes) 2024-09-17T23:30:11.377 INFO:teuthology.orchestra.run.smithi139.stdout:2960 read 0x8e79c00 thru 0x8e7a3a9 (0x7aa bytes) 2024-09-17T23:30:11.377 INFO:teuthology.orchestra.run.smithi139.stdout:2962 punch from 0x3b76800 to 0x3b7e600, (0x7e00 bytes) 2024-09-17T23:30:11.378 INFO:teuthology.orchestra.run.smithi139.stdout:2964 punch from 0x852d600 to 0x8530600, (0x3000 bytes) 2024-09-17T23:30:11.378 INFO:teuthology.orchestra.run.smithi139.stdout:2966 write 0x8a79000 thru 0x8a86a38 (0xda39 bytes) 2024-09-17T23:30:11.386 INFO:teuthology.orchestra.run.smithi139.stdout:2969 trunc from 0x9529200 to 0x24b0400 2024-09-17T23:30:11.413 INFO:teuthology.orchestra.run.smithi139.stdout:2974 write 0x1f73600 thru 0x1f7e674 (0xb075 bytes) 2024-09-17T23:30:11.417 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:11.416+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:11.434 INFO:teuthology.orchestra.run.smithi139.stdout:2975 write 0x84d9200 thru 0x84e295b (0x975c bytes) 2024-09-17T23:30:11.441 INFO:teuthology.orchestra.run.smithi139.stdout:2976 trunc from 0x84e2a00 to 0xd8da000 2024-09-17T23:30:11.453 INFO:teuthology.orchestra.run.smithi139.stdout:2977 read 0xa662a00 thru 0xa66e767 (0xbd68 bytes) 2024-09-17T23:30:11.454 INFO:teuthology.orchestra.run.smithi139.stdout:2978 punch from 0x7b75e00 to 0x7b83800, (0xda00 bytes) 2024-09-17T23:30:11.455 INFO:teuthology.orchestra.run.smithi139.stdout:2982 read 0x97f7a00 thru 0x9803893 (0xbe94 bytes) 2024-09-17T23:30:11.455 INFO:teuthology.orchestra.run.smithi139.stdout:2983 read 0xb32ac00 thru 0xb33427f (0x9680 bytes) 2024-09-17T23:30:11.455 INFO:teuthology.orchestra.run.smithi139.stdout:2985 read 0x5a9bc00 thru 0x5aa2d91 (0x7192 bytes) 2024-09-17T23:30:11.456 INFO:teuthology.orchestra.run.smithi139.stdout:2986 punch from 0x8b29600 to 0x8b2de00, (0x4800 bytes) 2024-09-17T23:30:11.456 INFO:teuthology.orchestra.run.smithi139.stdout:2987 write 0xc174e00 thru 0xc17c7c3 (0x79c4 bytes) 2024-09-17T23:30:11.456 INFO:teuthology.orchestra.run.smithi139.stdout:2990 read 0x2455000 thru 0x245ba7b (0x6a7c bytes) 2024-09-17T23:30:11.456 INFO:teuthology.orchestra.run.smithi139.stdout:2991 read 0x26fc400 thru 0x27052fa (0x8efb bytes) 2024-09-17T23:30:11.458 INFO:teuthology.orchestra.run.smithi139.stdout:2992 trunc from 0xd8da000 to 0xeaad000 2024-09-17T23:30:11.472 INFO:teuthology.orchestra.run.smithi139.stdout:2994 punch from 0xd53d200 to 0xd54ba00, (0xe800 bytes) 2024-09-17T23:30:11.473 INFO:teuthology.orchestra.run.smithi139.stdout:2995 write 0x4d8f200 thru 0x4d919ba (0x27bb bytes) 2024-09-17T23:30:11.474 INFO:teuthology.orchestra.run.smithi139.stdout:3000 read 0x466ec00 thru 0x4675d1b (0x711c bytes) 2024-09-17T23:30:11.475 INFO:teuthology.orchestra.run.smithi139.stdout:3003 write 0x1b43200 thru 0x1b44577 (0x1378 bytes) 2024-09-17T23:30:11.477 INFO:teuthology.orchestra.run.smithi139.stdout:3004 punch from 0x92cc200 to 0x92dac00, (0xea00 bytes) 2024-09-17T23:30:11.477 INFO:teuthology.orchestra.run.smithi139.stdout:3009 write 0x7029c00 thru 0x703713d (0xd53e bytes) 2024-09-17T23:30:11.477 INFO:teuthology.orchestra.run.smithi139.stdout:3011 read 0x4b1dc00 thru 0x4b2b52b (0xd92c bytes) 2024-09-17T23:30:11.478 INFO:teuthology.orchestra.run.smithi139.stdout:3016 read 0x2cafc00 thru 0x2cb9628 (0x9a29 bytes) 2024-09-17T23:30:11.478 INFO:teuthology.orchestra.run.smithi139.stdout:3017 read 0x8d5ea00 thru 0x8d6b2d5 (0xc8d6 bytes) 2024-09-17T23:30:11.479 INFO:teuthology.orchestra.run.smithi139.stdout:3018 clone 22 order 21 su 131072 sc 15 2024-09-17T23:30:11.480 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:11.479+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:11.492 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:11.492 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:11.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:11.504 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:11.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:11.522 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:11.538 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:11.538 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:11.556 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:11.557 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:11.574 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:11.574 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:11.581 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:11.580+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:11.591 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:11.610 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:11.611 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:11.611 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:11.611 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:11.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:11.682 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:11.680+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:11.726 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:11.727 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:11.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:11.783 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:11.781+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:11.884 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:11.882+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:11.885 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:11.885 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:11.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:11.885 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:11.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 45.007511827000002, 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:11.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:11.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:11.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.892 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991886696999998, 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:11.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:11.898 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991849041000002, 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:11.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:11.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:11.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991831609999998, 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:11.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:11.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991796678999997, 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:11.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:11.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:11.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.920 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991765258999997, 2024-09-17T23:30:11.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:11.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:11.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991748633, 2024-09-17T23:30:11.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:11.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:11.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991726939000003, 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:11.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 44.991715221, 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:11.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:11.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.427512878000002, 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:30:11.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:30:11.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:30:11.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.950 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.427490412000001, 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:30:11.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.955 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3262 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.420045633000001, 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045783590999999998, 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3262 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.462212+0000", 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463817+0000", 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00160411 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463821+0000", 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6670000000000004e-06 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463838+0000", 2024-09-17T23:30:11.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6898999999999999e-05 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.463848+0000", 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507787+0000", 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043939260000000001 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507895+0000", 2024-09-17T23:30:11.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010859899999999999 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.507996+0000", 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100587 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:11.960 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.410152128, 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:11.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:30:11.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.408244234000001, 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.965 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:30:11.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.969 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.406259164000002, 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.973 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.406249569, 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:11.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:30:11.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12611 3.18 3:1c8edc2b:::rbd_data.115262cdab6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.372938849000001, 2024-09-17T23:30:11.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.070059079999999996, 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12611 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509321+0000", 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-06 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553867+0000", 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044545920000000003 2024-09-17T23:30:11.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553893+0000", 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6094000000000001e-05 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553906+0000", 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3079e-05 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553972+0000", 2024-09-17T23:30:11.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6630000000000004e-05 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.554090+0000", 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117364 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.555383+0000", 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001293188 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.577545+0000", 2024-09-17T23:30:11.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.022161943 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579298+0000", 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017536349999999999 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579346+0000", 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7858000000000002e-05 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579378+0000", 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2011000000000002e-05 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12614 3.2 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.328318067000001, 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.100611516, 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12614 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553942+0000", 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.477e-06 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599428+0000", 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045486208 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599454+0000", 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6551999999999999e-05 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599467+0000", 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2339e-05 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599548+0000", 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599660+0000", 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011209699999999999 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.601086+0000", 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014260550000000001 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.612776+0000", 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011690010000000001 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654410+0000", 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.041634517000000003 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654486+0000", 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6260999999999994e-05 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654552+0000", 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5086999999999995e-05 2024-09-17T23:30:11.989 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4968.0:12613 3.9 e48/19 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head v 48'120, mlcod=45'117)", 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.284059973000002, 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049712148999999997, 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598199+0000", 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-07 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637181+0000", 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.038982528000000002 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637209+0000", 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7317e-05 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637219+0000", 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0372e-05 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637307+0000", 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7867000000000004e-05 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637345+0000", 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7735999999999999e-05 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647845+0000", 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010500012 2024-09-17T23:30:11.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647910+0000", 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5662999999999998e-05 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12676 3.1b 3:d8a99671:::rbd_data.115262cdab6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.277815359000002, 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.074765842999999998, 2024-09-17T23:30:11.994 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12676 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604445+0000", 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1399999999999998e-06 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638570+0000", 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.034124814000000003 2024-09-17T23:30:11.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638586+0000", 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6460000000000002e-05 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638611+0000", 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5216e-05 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638676+0000", 2024-09-17T23:30:11.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5048999999999993e-05 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638792+0000", 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115233 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.640186+0000", 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013940090000000001 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:11.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657756+0000", 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017570240000000001 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679079+0000", 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.021323312000000001 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679150+0000", 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1086000000000003e-05 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:11.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679209+0000", 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8284e-05 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.000 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:12.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:12.002 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:11.999+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:12.102 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:12.100+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:12.112 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.1 pg_upmap_items mapping to [3->2,5->0] 2024-09-17T23:30:12.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:30:12.157 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:12.157 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:12.157 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:12.157 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:12.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:12.194 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone21 intact 2024-09-17T23:30:12.211 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:12.209+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:12.275 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:12.275 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:12.276 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:12.276 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:12.276 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:12.284 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:12.303 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:12.302+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:12.405 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:12.403+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1984 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1814528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.939671666000002, 2024-09-17T23:30:12.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049003441000000002, 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1984 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:12.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492103+0000", 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1540000000000001e-06 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492107+0000", 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0249999999999996e-06 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492109+0000", 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540902+0000", 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048793087999999998 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540995+0000", 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2534000000000005e-05 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541105+0000", 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000109815 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.440 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1985 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1823744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.441 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:12.441 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.939660371999999, 2024-09-17T23:30:12.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049142263999999998, 2024-09-17T23:30:12.441 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1985 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493825+0000", 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017121879999999999 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493829+0000", 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2509999999999996e-06 2024-09-17T23:30:12.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493836+0000", 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000002e-06 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493840+0000", 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3220000000000003e-06 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541141+0000", 2024-09-17T23:30:12.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047301395000000003 2024-09-17T23:30:12.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541190+0000", 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8819999999999997e-05 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541255+0000", 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4587999999999999e-05 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.446 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2107 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1105920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.937798207, 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048932067000000003, 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2107 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.22e-07 2024-09-17T23:30:12.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493977+0000", 2024-09-17T23:30:12.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:30:12.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493984+0000", 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0920000000000003e-06 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493989+0000", 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0370000000000002e-06 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542786+0000", 2024-09-17T23:30:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797034000000003 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542841+0000", 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5781e-05 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542907+0000", 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5362000000000002e-05 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2108 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1115136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.937777677, 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049148982000000001, 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2108 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493996+0000", 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.03e-07 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493998+0000", 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.187e-06 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494004+0000", 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2219999999999999e-06 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494009+0000", 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6870000000000002e-06 2024-09-17T23:30:12.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542954+0000", 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048945206999999998 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543043+0000", 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9494999999999996e-05 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543144+0000", 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100781 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:12.457 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.937757687000001, 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:12.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.937737806999998, 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:12.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.937716268999999, 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.467 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2117 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1198080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935853954000002, 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048875710000000003, 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2117 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:30:12.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495920+0000", 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.105e-06 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495924+0000", 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.269e-06 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495926+0000", 2024-09-17T23:30:12.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5830000000000001e-06 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544618+0000", 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048691940000000003 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544704+0000", 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5581000000000006e-05 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544795+0000", 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1020999999999995e-05 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2118 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1207296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935843747, 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.04902596, 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2118 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495931+0000", 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.139e-06 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495934+0000", 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1870000000000001e-06 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495936+0000", 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.542e-06 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544842+0000", 2024-09-17T23:30:12.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048905954000000001 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544893+0000", 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0927999999999998e-05 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544955+0000", 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999000000000001e-05 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935833670999997, 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:12.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935823554999999, 2024-09-17T23:30:12.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:12.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935811325000003, 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.490 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.494 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935800755999999, 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935790867999998, 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:12.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:12.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935780586, 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:12.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:12.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:12.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:12.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.510 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.935764699000003, 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:12.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:12.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2129 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1308672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.933856788, 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048890621000000002, 2024-09-17T23:30:12.517 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2129 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.518 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:12.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5100000000000001e-07 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497918+0000", 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4670000000000001e-06 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497922+0000", 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497924+0000", 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6390000000000001e-06 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546622+0000", 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048697427000000001 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546706+0000", 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4306000000000002e-05 2024-09-17T23:30:12.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546807+0000", 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100851 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.396839382, 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:12.522 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:12.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:12.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:12.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:12.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.394769996000001, 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:12.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:12.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:12.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.062902628, 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:12.541 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.542 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.542 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.542 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:12.543 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:12.544 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:12.541+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:12.575 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T23:30:12.644 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:12.642+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:12.656 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:12.656 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:12.656 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:12.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:12.745 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:12.743+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:12.785 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:12.786 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:12.795 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:12.846 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:12.845+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:12.947 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:12.945+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:12.948 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.196134906000001, 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.196102543999999, 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:12.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:12.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.196081362000001, 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:12.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.196054799999999, 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.963 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:12.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.196032627000001, 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.967 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:12.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:12.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.971 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.196012714999998, 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:12.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:12.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:12.976 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.195966039000002, 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:12.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.195939290999998, 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.985 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.195911203000001, 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:12.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.195890903999999, 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:12.990 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:12.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:12.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.994 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 19.369092377000001, 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:12.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:12.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:12.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.000 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3143 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8651427070000004, 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0144227690000001, 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3143 2024-09-17T23:30:13.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080796+0000", 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2099999999999995e-07 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080801+0000", 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6909999999999996e-06 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080822+0000", 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1128000000000002e-05 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080830+0000", 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8280000000000003e-06 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1292999999999997e-05 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3889999999999999e-06 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083687+0000", 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027885470000000002 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083696+0000", 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3959999999999995e-06 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095020+0000", 2024-09-17T23:30:13.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011324347 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095106+0000", 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5352000000000005e-05 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095218+0000", 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111877 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3144 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.86508816, 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014824425, 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3144 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080850+0000", 2024-09-17T23:30:13.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080853+0000", 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2509999999999999e-06 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080861+0000", 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7840000000000004e-06 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080866+0000", 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080962+0000", 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5360999999999997e-05 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083738+0000", 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00276998 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083747+0000", 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1139999999999995e-06 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095514+0000", 2024-09-17T23:30:13.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011767071 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095579+0000", 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4732000000000003e-05 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095674+0000", 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4726000000000001e-05 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.012 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3145 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8650622969999997, 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014984876, 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3145 2024-09-17T23:30:13.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080876+0000", 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080878+0000", 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-06 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080886+0000", 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3599999999999996e-06 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8559999999999998e-06 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081004+0000", 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112568 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081009+0000", 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:30:13.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083807+0000", 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002797538 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083825+0000", 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8416000000000001e-05 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095733+0000", 2024-09-17T23:30:13.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011907194 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095780+0000", 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7942000000000003e-05 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095860+0000", 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9721999999999993e-05 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3146 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8650395800000004, 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015105355, 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3146 2024-09-17T23:30:13.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7e-07 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080900+0000", 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8679999999999999e-06 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080906+0000", 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7440000000000004e-06 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080911+0000", 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081059+0000", 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014868999999999999 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081067+0000", 2024-09-17T23:30:13.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5650000000000004e-06 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083866+0000", 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002799008 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083871+0000", 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9509999999999997e-06 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095889+0000", 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012018283 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095931+0000", 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1885999999999998e-05 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096003+0000", 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2390000000000003e-05 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.024 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3147 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8650214500000004, 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015262753, 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3147 2024-09-17T23:30:13.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080917+0000", 2024-09-17T23:30:13.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080918+0000", 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7999999999999999e-06 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080924+0000", 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5310000000000001e-06 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080928+0000", 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081109+0000", 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018069200000000001 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081115+0000", 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7490000000000004e-06 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083903+0000", 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027881249999999998 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083908+0000", 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096026+0000", 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0121184919999999 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096065+0000", 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8566999999999999e-05 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096179+0000", 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011409900000000001 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3148 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8649853780000001, 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015338839, 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3148 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080953+0000", 2024-09-17T23:30:13.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4099999999999999e-07 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080955+0000", 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.21e-06 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080963+0000", 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7910000000000005e-06 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9350000000000002e-06 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081149+0000", 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018104800000000001 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081152+0000", 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083933+0000", 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027811630000000001 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083937+0000", 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0609999999999997e-06 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096195+0000", 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122576050000001 2024-09-17T23:30:13.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096232+0000", 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7592999999999997e-05 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096291+0000", 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8767999999999997e-05 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3149 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8649448099999999, 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154068810000001, 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3149 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2e-07 2024-09-17T23:30:13.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080995+0000", 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1299999999999999e-06 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081003+0000", 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4089999999999997e-06 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081008+0000", 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8940000000000001e-06 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081196+0000", 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018834400000000001 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081202+0000", 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083961+0000", 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027589340000000002 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083965+0000", 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0029999999999996e-06 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096305+0000", 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012340311 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096343+0000", 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7654000000000002e-05 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096400+0000", 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6876999999999998e-05 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.042 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3150 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8649237870000004, 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154925990000001, 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3150 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081016+0000", 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.916e-06 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081022+0000", 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.66e-06 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081026+0000", 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3490000000000002e-06 2024-09-17T23:30:13.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081221+0000", 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000194599 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081223+0000", 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.694e-06 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083982+0000", 2024-09-17T23:30:13.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002758122 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083985+0000", 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9360000000000001e-06 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096413+0000", 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124277079999999 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:13.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096450+0000", 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6449999999999998e-05 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096506+0000", 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6820999999999997e-05 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4997.0:1730 3.1f 3:fa9e12c1:::rbd_data.1152ce7c866.0000000000000117:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.095427332000000004, 2024-09-17T23:30:13.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063375800999999995, 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4997", 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1359963294", 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1730 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850511+0000", 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850517+0000", 2024-09-17T23:30:13.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850522+0000", 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850526+0000", 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:13.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913749+0000", 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063222924 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913886+0000", 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013731999999999999 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:13.052 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:13.054 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:13.052+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:13.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:13.116 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:30:13.142 INFO:teuthology.orchestra.run.smithi139.stdout:3019 write 0xb6aa400 thru 0xb6b50da (0xacdb bytes) 2024-09-17T23:30:13.144 INFO:teuthology.orchestra.run.smithi139.stdout:3021 read 0x6d89c00 thru 0x6d9853e (0xe93f bytes) 2024-09-17T23:30:13.146 INFO:teuthology.orchestra.run.smithi139.stdout:3022 write 0xcad4a00 thru 0xcadfdbb (0xb3bc bytes) 2024-09-17T23:30:13.147 INFO:teuthology.orchestra.run.smithi139.stdout:3023 punch from 0xea3a600 to 0xea3aa00, (0x400 bytes) 2024-09-17T23:30:13.154 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:13.152+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:13.162 INFO:teuthology.orchestra.run.smithi139.stdout:3024 punch from 0x902d000 to 0x9031a00, (0x4a00 bytes) 2024-09-17T23:30:13.168 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:13.168 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:13.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:13.180 INFO:teuthology.orchestra.run.smithi139.stdout:3026 trunc from 0xeaad000 to 0x275e000 2024-09-17T23:30:13.226 INFO:teuthology.orchestra.run.smithi139.stdout:3027 read 0x55c000 thru 0x56b44d (0xf44e bytes) 2024-09-17T23:30:13.226 INFO:teuthology.orchestra.run.smithi139.stdout:3028 punch from 0x26f9800 to 0x26fe200, (0x4a00 bytes) 2024-09-17T23:30:13.227 INFO:teuthology.orchestra.run.smithi139.stdout:3029 punch from 0xe89000 to 0xe8c400, (0x3400 bytes) 2024-09-17T23:30:13.228 INFO:teuthology.orchestra.run.smithi139.stdout:3030 read 0x1ee0a00 thru 0x1eeb19b (0xa79c bytes) 2024-09-17T23:30:13.228 INFO:teuthology.orchestra.run.smithi139.stdout:3031 read 0xfb6c00 thru 0xfbf07d (0x847e bytes) 2024-09-17T23:30:13.254 INFO:teuthology.orchestra.run.smithi139.stdout:3033 write 0xb9ab400 thru 0xb9b8539 (0xd13a bytes) 2024-09-17T23:30:13.254 INFO:teuthology.orchestra.run.smithi139.stdout:3035 write 0x982e600 thru 0x9834da1 (0x67a2 bytes) 2024-09-17T23:30:13.254 INFO:teuthology.orchestra.run.smithi139.stdout:3036 write 0xaccac00 thru 0xaccd0a3 (0x24a4 bytes) 2024-09-17T23:30:13.255 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:13.253+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:13.267 INFO:teuthology.orchestra.run.smithi139.stdout:3037 trunc from 0xb9b8600 to 0x3aa000 2024-09-17T23:30:13.319 INFO:teuthology.orchestra.run.smithi139.stdout:3038 read 0x1b5200 thru 0x1c12d7 (0xc0d8 bytes) 2024-09-17T23:30:13.320 INFO:teuthology.orchestra.run.smithi139.stdout:3041 punch from 0xd0e00 to 0xdfa00, (0xec00 bytes) 2024-09-17T23:30:13.334 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:13.335 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:13.341 INFO:teuthology.orchestra.run.smithi139.stdout:3042 write 0x656f200 thru 0x657cfcd (0xddce bytes) 2024-09-17T23:30:13.341 INFO:teuthology.orchestra.run.smithi139.stdout:3043 read 0x8a6c00 thru 0x8b11ce (0xa5cf bytes) 2024-09-17T23:30:13.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:13.350 INFO:teuthology.orchestra.run.smithi139.stdout:3044 trunc from 0x657d000 to 0xe430800 2024-09-17T23:30:13.368 INFO:teuthology.orchestra.run.smithi139.stdout:3046 read 0x692a200 thru 0x6934bb2 (0xa9b3 bytes) 2024-09-17T23:30:13.369 INFO:teuthology.orchestra.run.smithi139.stdout:3047 read 0x1798000 thru 0x17989c7 (0x9c8 bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3048 write 0xbdb6600 thru 0xbdbe8bf (0x82c0 bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3050 read 0xc5ac400 thru 0xc5afc2e (0x382f bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3051 write 0xa6b1000 thru 0xa6b399c (0x299d bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3054 write 0x4c69200 thru 0x4c6fbdd (0x69de bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3055 read 0x47b6800 thru 0x47bb870 (0x5071 bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3056 read 0xa002e00 thru 0xa00df01 (0xb102 bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3058 read 0x3cf6800 thru 0x3cfecf3 (0x84f4 bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3060 read 0x7145600 thru 0x714c571 (0x6f72 bytes) 2024-09-17T23:30:13.370 INFO:teuthology.orchestra.run.smithi139.stdout:3061 punch from 0x7d4ac00 to 0x7d56a00, (0xbe00 bytes) 2024-09-17T23:30:13.371 INFO:teuthology.orchestra.run.smithi139.stdout:3063 trunc from 0xe430800 to 0xd6b3400 2024-09-17T23:30:13.387 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:13.386+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:13.395 INFO:teuthology.orchestra.run.smithi139.stdout:3064 read 0x509be00 thru 0x509ca09 (0xc0a bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3065 write 0xe24d400 thru 0xe255de7 (0x89e8 bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3067 write 0xc566e00 thru 0xc5737ea (0xc9eb bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3068 write 0x6433a00 thru 0x6436c1b (0x321c bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3069 read 0x2b5dc00 thru 0x2b68aa9 (0xaeaa bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3070 read 0x80fb000 thru 0x81095dd (0xe5de bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3072 write 0x2eb0200 thru 0x2ebbeeb (0xbcec bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3073 write 0x3c4b600 thru 0x3c5743e (0xbe3f bytes) 2024-09-17T23:30:13.414 INFO:teuthology.orchestra.run.smithi139.stdout:3075 write 0x9f9e800 thru 0x9faa25e (0xba5f bytes) 2024-09-17T23:30:13.415 INFO:teuthology.orchestra.run.smithi139.stdout:3078 write 0x4e8b800 thru 0x4e8bcbc (0x4bd bytes) 2024-09-17T23:30:13.415 INFO:teuthology.orchestra.run.smithi139.stdout:3079 write 0x178e00 thru 0x17d2a2 (0x44a3 bytes) 2024-09-17T23:30:13.415 INFO:teuthology.orchestra.run.smithi139.stdout:3081 read 0x2f7c800 thru 0x2f8b63b (0xee3c bytes) 2024-09-17T23:30:13.415 INFO:teuthology.orchestra.run.smithi139.stdout:3082 read 0x552d000 thru 0x5532362 (0x5363 bytes) 2024-09-17T23:30:13.416 INFO:teuthology.orchestra.run.smithi139.stdout:3083 read 0x8e94200 thru 0x8e9f395 (0xb196 bytes) 2024-09-17T23:30:13.416 INFO:teuthology.orchestra.run.smithi139.stdout:3084 read 0xd048a00 thru 0xd055946 (0xcf47 bytes) 2024-09-17T23:30:13.416 INFO:teuthology.orchestra.run.smithi139.stdout:3085 write 0x642f800 thru 0x6434646 (0x4e47 bytes) 2024-09-17T23:30:13.416 INFO:teuthology.orchestra.run.smithi139.stdout:3086 trunc from 0xe255e00 to 0xeb73000 2024-09-17T23:30:13.438 INFO:teuthology.orchestra.run.smithi139.stdout:3088 read 0xdde3600 thru 0xddf1be5 (0xe5e6 bytes) 2024-09-17T23:30:13.439 INFO:teuthology.orchestra.run.smithi139.stdout:3089 read 0x3d2000 thru 0x3db41f (0x9420 bytes) 2024-09-17T23:30:13.444 INFO:teuthology.orchestra.run.smithi139.stdout:3091 trunc from 0xeb73000 to 0xab80200 2024-09-17T23:30:13.457 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:13.455+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:13.459 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9550 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.165690680000001, 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056192745000000002, 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9550 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290751+0000", 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290757+0000", 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290762+0000", 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346584+0000", 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055821636000000001 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346864+0000", 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028079899999999998 2024-09-17T23:30:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346941+0000", 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6173000000000004e-05 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9552 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.165453765999999, 2024-09-17T23:30:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056146769999999999, 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9552 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8999999999999996e-07 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290994+0000", 2024-09-17T23:30:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3399999999999998e-06 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291001+0000", 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1910000000000003e-06 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291007+0000", 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346974+0000", 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055966828000000003 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347059+0000", 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5753000000000004e-05 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347131+0000", 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-17T23:30:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9604 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.165384393, 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05626722, 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9604 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291055+0000", 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291057+0000", 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291064+0000", 2024-09-17T23:30:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291070+0000", 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347164+0000", 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056094887000000003 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347250+0000", 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5552999999999999e-05 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347321+0000", 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1310999999999995e-05 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9606 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.165118701000001, 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05619619, 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9606 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1900000000000002e-07 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291324+0000", 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6619999999999998e-06 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291333+0000", 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291339+0000", 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.477e-06 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347352+0000", 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056012987 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347434+0000", 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1569000000000005e-05 2024-09-17T23:30:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347516+0000", 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2365000000000002e-05 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9608 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.165088539999999, 2024-09-17T23:30:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056951664999999999, 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9608 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291351+0000", 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8299999999999997e-07 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291353+0000", 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.396e-06 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291360+0000", 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.199e-06 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291367+0000", 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2230000000000001e-06 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347546+0000", 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056178616000000001 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348232+0000", 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068600200000000001 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348302+0000", 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9646000000000004e-05 2024-09-17T23:30:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9610 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.164925849999999, 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056957865000000003, 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9610 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291517+0000", 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.393e-06 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291526+0000", 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0129999999999999e-06 2024-09-17T23:30:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291532+0000", 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8430000000000004e-06 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348331+0000", 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056798279 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348405+0000", 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4418000000000005e-05 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348470+0000", 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5355e-05 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9613 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.164855805999998, 2024-09-17T23:30:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057056663000000001, 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9613 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291585+0000", 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291593+0000", 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291596+0000", 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.269e-06 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348498+0000", 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056902369000000001 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348573+0000", 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4962000000000005e-05 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348639+0000", 2024-09-17T23:30:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5874e-05 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9615 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.164692087999999, 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057044395999999997, 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9615 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291747+0000", 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291749+0000", 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291754+0000", 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410000000000003e-06 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291757+0000", 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4520000000000002e-06 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348665+0000", 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056908513000000001 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348732+0000", 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6180999999999995e-05 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348791+0000", 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9129000000000003e-05 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9620 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.164537735, 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057063968999999999, 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9620 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291909+0000", 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5610000000000001e-06 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291916+0000", 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0679999999999999e-06 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291923+0000", 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-17T23:30:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348817+0000", 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056893959000000001 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348898+0000", 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0377999999999995e-05 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348965+0000", 2024-09-17T23:30:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7146999999999999e-05 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9622 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.16447926, 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057231199000000003, 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9622 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291960+0000", 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291962+0000", 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7939999999999998e-06 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291970+0000", 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5009999999999998e-06 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291975+0000", 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999999e-06 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349015+0000", 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057039443000000002 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349104+0000", 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9791999999999997e-05 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349190+0000", 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6055000000000002e-05 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9716 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.160333360999999, 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056085545000000001, 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9716 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296106+0000", 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296108+0000", 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296114+0000", 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7830000000000004e-06 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296119+0000", 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-17T23:30:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352074+0000", 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055955619999999998 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352121+0000", 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6857999999999998e-05 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352191+0000", 2024-09-17T23:30:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9639000000000002e-05 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9722 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.160274442999999, 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056209310999999998, 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9722 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296362+0000", 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019768800000000001 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296366+0000", 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9419999999999997e-06 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296375+0000", 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6309999999999998e-06 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296392+0000", 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6758000000000001e-05 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352221+0000", 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055828950000000002 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352299+0000", 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8375999999999996e-05 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352373+0000", 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3966000000000003e-05 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9723 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.160035534999999, 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056145054999999999, 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9723 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296404+0000", 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8800000000000002e-07 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296411+0000", 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296419+0000", 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129999999999993e-06 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296424+0000", 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5879999999999997e-06 2024-09-17T23:30:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352405+0000", 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055980769 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352479+0000", 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4018999999999997e-05 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352548+0000", 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8997000000000004e-05 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9729 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.159980353999998, 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056272003000000001, 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9729 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296459+0000", 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2799999999999996e-07 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296461+0000", 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439999999999998e-06 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296468+0000", 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296472+0000", 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352583+0000", 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056111017999999999 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352665+0000", 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1148000000000003e-05 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352730+0000", 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5477999999999999e-05 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9731 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.159958969000002, 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056506531999999998, 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9731 2024-09-17T23:30:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297569+0000", 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001089394 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297574+0000", 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7260000000000002e-06 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297583+0000", 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4169999999999998e-06 2024-09-17T23:30:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297590+0000", 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352775+0000", 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055185161000000003 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352867+0000", 2024-09-17T23:30:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2090999999999996e-05 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352986+0000", 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011883300000000001 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9743 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.158689355, 2024-09-17T23:30:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055972380000000002, 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9743 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297750+0000", 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297752+0000", 2024-09-17T23:30:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297760+0000", 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1839999999999999e-06 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297766+0000", 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353551+0000", 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055784783999999997 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353645+0000", 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3760999999999999e-05 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353721+0000", 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6907000000000001e-05 2024-09-17T23:30:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9744 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.158664486999999, 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056155391999999998, 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9744 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297775+0000", 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297777+0000", 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298431+0000", 2024-09-17T23:30:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065376599999999996 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298441+0000", 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8709999999999992e-06 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353773+0000", 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055332332999999997 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353856+0000", 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3122999999999997e-05 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353929+0000", 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3183999999999996e-05 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9795 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 4677632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.156280662, 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056138625999999997, 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9795 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301567+0000", 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001409492 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301573+0000", 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2159999999999997e-06 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301582+0000", 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5619999999999993e-06 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301588+0000", 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-17T23:30:13.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356166+0000", 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054577696000000002 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356253+0000", 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6526000000000005e-05 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356296+0000", 2024-09-17T23:30:13.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3918999999999997e-05 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12612 3.13 3:ccf171c8:::rbd_data.115262cdab6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.945747408999999, 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063461453000000001, 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12612 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510693+0000", 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9e-06 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539114+0000", 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028420697000000002 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539145+0000", 2024-09-17T23:30:13.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1488000000000001e-05 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539159+0000", 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3737000000000001e-05 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539226+0000", 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7572999999999998e-05 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539337+0000", 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110849 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.540857+0000", 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001520043 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.571640+0000", 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030782961000000001 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574080+0000", 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024394529999999998 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574120+0000", 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0459e-05 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574153+0000", 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292999999999997e-05 2024-09-17T23:30:13.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12613 3.9 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.917249407, 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.13090919500000001, 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:13.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12613 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:13.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539191+0000", 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.59e-06 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.595998+0000", 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056807745999999999 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596021+0000", 2024-09-17T23:30:13.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2574000000000001e-05 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596030+0000", 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1670000000000005e-06 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596150+0000", 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119688 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:13.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596301+0000", 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015097400000000001 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598103+0000", 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001802281 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.610230+0000", 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012126468 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.669969+0000", 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059739318999999999 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670041+0000", 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1834000000000004e-05 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670098+0000", 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7553999999999998e-05 2024-09-17T23:30:13.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:13.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:13.554 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:13.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3092 punch from 0x4d27000 to 0x4d36600, (0xf600 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3093 punch from 0x6b29e00 to 0x6b32a00, (0x8c00 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3096 write 0xc787600 thru 0xc78ffc4 (0x89c5 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3097 write 0x9574200 thru 0x957bb81 (0x7982 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3101 write 0xc5bf600 thru 0xc5ca210 (0xac11 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3103 write 0xad51a00 thru 0xad5600d (0x460e bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3105 write 0x9dd9000 thru 0x9de43f9 (0xb3fa bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3115 trunc from 0xc790000 to 0x955aa00 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3117 read 0x58b3e00 thru 0x58bc68c (0x888d bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3119 punch from 0x3f75800 to 0x3f82000, (0xc800 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3120 write 0x67a0800 thru 0x67a63e7 (0x5be8 bytes) 2024-09-17T23:30:13.557 INFO:teuthology.orchestra.run.smithi139.stdout:3121 write 0x104b800 thru 0x1053ec9 (0x86ca bytes) 2024-09-17T23:30:13.558 INFO:teuthology.orchestra.run.smithi139.stdout:3122 punch from 0x31f0c00 to 0x31fa200, (0x9600 bytes) 2024-09-17T23:30:13.558 INFO:teuthology.orchestra.run.smithi139.stdout:3123 trunc from 0x955aa00 to 0x33d7e00 2024-09-17T23:30:13.564 INFO:teuthology.orchestra.run.smithi139.stdout:3124 write 0x6c66800 thru 0x6c75d2e (0xf52f bytes) 2024-09-17T23:30:13.571 INFO:teuthology.orchestra.run.smithi139.stdout:3126 punch from 0x5fdd200 to 0x5fe8800, (0xb600 bytes) 2024-09-17T23:30:13.592 INFO:teuthology.orchestra.run.smithi139.stdout:3127 write 0xeaa9400 thru 0xeab3ce3 (0xa8e4 bytes) 2024-09-17T23:30:13.593 INFO:teuthology.orchestra.run.smithi139.stdout:3129 write 0xb6bca00 thru 0xb6cbe0c (0xf40d bytes) 2024-09-17T23:30:13.596 INFO:teuthology.orchestra.run.smithi139.stdout:3130 trunc from 0xeab3e00 to 0xc1c8c00 2024-09-17T23:30:13.597 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:13.595+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:13.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:13.618 INFO:teuthology.orchestra.run.smithi139.stdout:3131 write 0x45da400 thru 0x45dea0a (0x460b bytes) 2024-09-17T23:30:13.634 INFO:teuthology.orchestra.run.smithi139.stdout:3133 write 0xe2a8c00 thru 0xe2aa8aa (0x1cab bytes) 2024-09-17T23:30:13.655 INFO:teuthology.orchestra.run.smithi139.stdout:3137 trunc from 0xe2aaa00 to 0x337a000 2024-09-17T23:30:13.660 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:13.658+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:13.685 INFO:teuthology.orchestra.run.smithi139.stdout:3138 trunc from 0x337a000 to 0xb6f8200 2024-09-17T23:30:13.696 INFO:teuthology.orchestra.run.smithi139.stdout:3139 trunc from 0xb6f8200 to 0xbfc8400 2024-09-17T23:30:13.709 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:13.709 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:13.709 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:13.710 INFO:teuthology.orchestra.run.smithi139.stdout:3142 punch from 0xa0e8200 to 0xa0f3800, (0xb600 bytes) 2024-09-17T23:30:13.712 INFO:teuthology.orchestra.run.smithi139.stdout:3144 punch from 0x4ea9200 to 0x4eb8000, (0xee00 bytes) 2024-09-17T23:30:13.712 INFO:teuthology.orchestra.run.smithi139.stdout:3145 write 0x520b600 thru 0x520fd38 (0x4739 bytes) 2024-09-17T23:30:13.712 INFO:teuthology.orchestra.run.smithi139.stdout:3146 write 0x6480200 thru 0x648e32c (0xe12d bytes) 2024-09-17T23:30:13.712 INFO:teuthology.orchestra.run.smithi139.stdout:3148 read 0xbddda00 thru 0xbde4903 (0x6f04 bytes) 2024-09-17T23:30:13.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:13.722 INFO:teuthology.orchestra.run.smithi139.stdout:3149 trunc from 0xbfc8400 to 0x40f9600 2024-09-17T23:30:13.750 INFO:teuthology.orchestra.run.smithi139.stdout:3150 trunc from 0x40f9600 to 0xd307a00 2024-09-17T23:30:13.759 INFO:teuthology.orchestra.run.smithi139.stdout:3152 read 0x9e5a400 thru 0x9e66730 (0xc331 bytes) 2024-09-17T23:30:13.762 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:13.761+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:13.766 INFO:teuthology.orchestra.run.smithi139.stdout:3157 write 0x907fe00 thru 0x908939f (0x95a0 bytes) 2024-09-17T23:30:13.766 INFO:teuthology.orchestra.run.smithi139.stdout:3158 punch from 0x3bfb000 to 0x3c00000, (0x5000 bytes) 2024-09-17T23:30:13.767 INFO:teuthology.orchestra.run.smithi139.stdout:3161 punch from 0x5210600 to 0x5212400, (0x1e00 bytes) 2024-09-17T23:30:13.767 INFO:teuthology.orchestra.run.smithi139.stdout:3166 read 0xa22b600 thru 0xa22c961 (0x1362 bytes) 2024-09-17T23:30:13.767 INFO:teuthology.orchestra.run.smithi139.stdout:3167 write 0x7df2c00 thru 0x7df4ddc (0x21dd bytes) 2024-09-17T23:30:13.767 INFO:teuthology.orchestra.run.smithi139.stdout:3169 read 0x5989e00 thru 0x59920db (0x82dc bytes) 2024-09-17T23:30:13.768 INFO:teuthology.orchestra.run.smithi139.stdout:3172 write 0x70d5800 thru 0x70d6e98 (0x1699 bytes) 2024-09-17T23:30:13.772 INFO:teuthology.orchestra.run.smithi139.stdout:3173 trunc from 0xd307a00 to 0xaaa2400 2024-09-17T23:30:13.772 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:13.772 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:13.794 INFO:teuthology.orchestra.run.smithi139.stdout:3175 trunc from 0xaaa2400 to 0xa816000 2024-09-17T23:30:13.794 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:13.795 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:13.805 INFO:teuthology.orchestra.run.smithi139.stdout:3179 write 0x60ada00 thru 0x60b582d (0x7e2e bytes) 2024-09-17T23:30:13.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:13.808 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:13.812 INFO:teuthology.orchestra.run.smithi139.stdout:3181 read 0x3c4000 thru 0x3c775e (0x375f bytes) 2024-09-17T23:30:13.812 INFO:teuthology.orchestra.run.smithi139.stdout:3183 punch from 0x3300600 to 0x330bc00, (0xb600 bytes) 2024-09-17T23:30:13.818 INFO:teuthology.orchestra.run.smithi139.stdout:3185 trunc from 0xa816000 to 0x52a5200 2024-09-17T23:30:13.827 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:13.827 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:13.834 INFO:teuthology.orchestra.run.smithi139.stdout:3186 read 0x4bb8400 thru 0x4bc5fee (0xdbef bytes) 2024-09-17T23:30:13.835 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:13.835 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:13.840 INFO:teuthology.orchestra.run.smithi139.stdout:3188 read 0x3444c00 thru 0x34534c3 (0xe8c4 bytes) 2024-09-17T23:30:13.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:13.844 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:13.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:13.846 INFO:teuthology.orchestra.run.smithi139.stdout:3189 trunc from 0x52a5200 to 0x988e400 2024-09-17T23:30:13.855 INFO:teuthology.orchestra.run.smithi139.stdout:3191 punch from 0x974da00 to 0x9753600, (0x5c00 bytes) 2024-09-17T23:30:13.856 INFO:teuthology.orchestra.run.smithi139.stdout:3192 trunc from 0x988e400 to 0x93f9c00 2024-09-17T23:30:13.861 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:13.862 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:13.862 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:13.860+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:13.873 INFO:teuthology.orchestra.run.smithi139.stdout:3197 read 0x60c4e00 thru 0x60d3946 (0xeb47 bytes) 2024-09-17T23:30:13.885 INFO:teuthology.orchestra.run.smithi139.stdout:3198 write 0xb453000 thru 0xb462482 (0xf483 bytes) 2024-09-17T23:30:13.891 INFO:teuthology.orchestra.run.smithi139.stdout:3201 write 0x725e000 thru 0x726b1b2 (0xd1b3 bytes) 2024-09-17T23:30:13.891 INFO:teuthology.orchestra.run.smithi139.stdout:3202 read 0xa9ce400 thru 0xa9ddd7e (0xf97f bytes) 2024-09-17T23:30:13.907 INFO:teuthology.orchestra.run.smithi139.stdout:3203 write 0xb800000 thru 0xb80a12a (0xa12b bytes) 2024-09-17T23:30:13.915 INFO:teuthology.orchestra.run.smithi139.stdout:3204 read 0x2419600 thru 0x242904f (0xfa50 bytes) 2024-09-17T23:30:13.916 INFO:teuthology.orchestra.run.smithi139.stdout:3208 write 0x4eaa00 thru 0x4f85fd (0xdbfe bytes) 2024-09-17T23:30:13.928 INFO:teuthology.orchestra.run.smithi139.stdout:3209 trunc from 0xb80a200 to 0x1d1e200 2024-09-17T23:30:13.946 INFO:teuthology.orchestra.run.smithi139.stdout:3210 punch from 0x2a8000 to 0x2b2e00, (0xae00 bytes) 2024-09-17T23:30:13.947 INFO:teuthology.orchestra.run.smithi139.stdout:3211 punch from 0x1a3200 to 0x1aa600, (0x7400 bytes) 2024-09-17T23:30:13.961 INFO:teuthology.orchestra.run.smithi139.stdout:3212 trunc from 0x1d1e200 to 0xcf35e00 2024-09-17T23:30:13.964 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:13.962+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:13.976 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:13.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:13.991 INFO:teuthology.orchestra.run.smithi139.stdout:3216 write 0xe0dcc00 thru 0xe0e9395 (0xc796 bytes) 2024-09-17T23:30:13.991 INFO:teuthology.orchestra.run.smithi139.stdout:3217 write 0x14f7600 thru 0x15003f7 (0x8df8 bytes) 2024-09-17T23:30:13.992 INFO:teuthology.orchestra.run.smithi139.stdout:3219 punch from 0xaf51e00 to 0xaf53000, (0x1200 bytes) 2024-09-17T23:30:13.992 INFO:teuthology.orchestra.run.smithi139.stdout:3220 read 0x36a4a00 thru 0x36a91b0 (0x47b1 bytes) 2024-09-17T23:30:13.993 INFO:teuthology.orchestra.run.smithi139.stdout:3222 read 0x2ef9000 thru 0x2ef90f7 (0xf8 bytes) 2024-09-17T23:30:13.993 INFO:teuthology.orchestra.run.smithi139.stdout:3223 write 0x87ce800 thru 0x87d32d1 (0x4ad2 bytes) 2024-09-17T23:30:13.993 INFO:teuthology.orchestra.run.smithi139.stdout:3224 clone 23 order 18 su 8192 sc 9 2024-09-17T23:30:14.064 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:14.063+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.135 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:14.189 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:14.187+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:14.201 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone22 from 0xe0e9400 (overlap 0x3aa000) to 0x0 2024-09-17T23:30:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.257 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:14.267 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:14.264+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:14.366 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:14.364+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:14.420 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.421 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:14.468 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:14.466+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:14.513 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T23:30:14.541 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.541 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:14.568 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:14.567+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:14.670 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:14.668+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:14.671 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.672 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:14.770 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:14.769+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:14.804 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.804 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.813 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:14.872 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:14.870+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:14.932 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:14.932 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:14.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:14.973 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:14.971+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:15.050 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.050 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.059 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:15.074 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:15.072+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:15.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:30:15.165 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.165 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:15.175 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:15.173+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:15.182 INFO:teuthology.orchestra.run.smithi139.stdout:3227 trunc from 0xe0e9400 to 0x6975000 2024-09-17T23:30:15.232 INFO:teuthology.orchestra.run.smithi139.stdout:3229 write 0x86ccc00 thru 0x86dbaf1 (0xeef2 bytes) 2024-09-17T23:30:15.233 INFO:teuthology.orchestra.run.smithi139.stdout:3231 punch from 0x892400 to 0x89fe00, (0xda00 bytes) 2024-09-17T23:30:15.245 INFO:teuthology.orchestra.run.smithi139.stdout:3233 trunc from 0x86dbc00 to 0x1d66400 2024-09-17T23:30:15.276 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:15.274+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:15.294 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.295 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:15.313 INFO:teuthology.orchestra.run.smithi139.stdout:3234 write 0x3d57e00 thru 0x3d5fc0f (0x7e10 bytes) 2024-09-17T23:30:15.314 INFO:teuthology.orchestra.run.smithi139.stdout:3235 punch from 0x2d06200 to 0x2d10a00, (0xa800 bytes) 2024-09-17T23:30:15.314 INFO:teuthology.orchestra.run.smithi139.stdout:3238 read 0x1ea0e00 thru 0x1ea5429 (0x462a bytes) 2024-09-17T23:30:15.340 INFO:teuthology.orchestra.run.smithi139.stdout:3240 write 0x701ca00 thru 0x7025279 (0x887a bytes) 2024-09-17T23:30:15.340 INFO:teuthology.orchestra.run.smithi139.stdout:3241 read 0x4fb0000 thru 0x4fb02d5 (0x2d6 bytes) 2024-09-17T23:30:15.369 INFO:teuthology.orchestra.run.smithi139.stdout:3242 write 0xc1bbe00 thru 0xc1bedda (0x2fdb bytes) 2024-09-17T23:30:15.369 INFO:teuthology.orchestra.run.smithi139.stdout:3243 write 0x8737600 thru 0x8739763 (0x2164 bytes) 2024-09-17T23:30:15.374 INFO:teuthology.orchestra.run.smithi139.stdout:3244 trunc from 0xc1bee00 to 0x7d63800 2024-09-17T23:30:15.377 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:15.375+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:15.404 INFO:teuthology.orchestra.run.smithi139.stdout:3247 read 0xd70a00 thru 0xd79439 (0x8a3a bytes) 2024-09-17T23:30:15.420 INFO:teuthology.orchestra.run.smithi139.stdout:3249 write 0x80ff600 thru 0x8104aaf (0x54b0 bytes) 2024-09-17T23:30:15.421 INFO:teuthology.orchestra.run.smithi139.stdout:3250 punch from 0x101f200 to 0x1026e00, (0x7c00 bytes) 2024-09-17T23:30:15.428 INFO:teuthology.orchestra.run.smithi139.stdout:3251 trunc from 0x8104c00 to 0xc1c5c00 2024-09-17T23:30:15.457 INFO:teuthology.orchestra.run.smithi139.stdout:3252 read 0xbbe5c00 thru 0xbbebcf6 (0x60f7 bytes) 2024-09-17T23:30:15.457 INFO:teuthology.orchestra.run.smithi139.stdout:3253 write 0x1c1aa00 thru 0x1c2642d (0xba2e bytes) 2024-09-17T23:30:15.458 INFO:teuthology.orchestra.run.smithi139.stdout:3257 read 0x1b65000 thru 0x1b7143f (0xc440 bytes) 2024-09-17T23:30:15.460 INFO:teuthology.orchestra.run.smithi139.stdout:3258 trunc from 0xc1c5c00 to 0xb4efe00 2024-09-17T23:30:15.468 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.468 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.477 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:15.476+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:15.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:15.494 INFO:teuthology.orchestra.run.smithi139.stdout:3262 punch from 0x2ef1c00 to 0x2eff000, (0xd400 bytes) 2024-09-17T23:30:15.495 INFO:teuthology.orchestra.run.smithi139.stdout:3263 write 0x19b5800 thru 0x19b99c1 (0x41c2 bytes) 2024-09-17T23:30:15.496 INFO:teuthology.orchestra.run.smithi139.stdout:3267 write 0xad3c600 thru 0xad41f1c (0x591d bytes) 2024-09-17T23:30:15.502 INFO:teuthology.orchestra.run.smithi139.stdout:3268 trunc from 0xb4efe00 to 0x6344c00 2024-09-17T23:30:15.538 INFO:teuthology.orchestra.run.smithi139.stdout:3270 trunc from 0x6344c00 to 0x6419a00 2024-09-17T23:30:15.561 INFO:teuthology.orchestra.run.smithi139.stdout:3274 trunc from 0x6419a00 to 0xca3b400 2024-09-17T23:30:15.577 INFO:teuthology.orchestra.run.smithi139.stdout:3276 punch from 0x30c1c00 to 0x30c2400, (0x800 bytes) 2024-09-17T23:30:15.578 INFO:teuthology.orchestra.run.smithi139.stdout:3277 read 0xb75cc00 thru 0xb7697b2 (0xcbb3 bytes) 2024-09-17T23:30:15.578 INFO:teuthology.orchestra.run.smithi139.stdout:3281 punch from 0x16b2800 to 0x16b2e00, (0x600 bytes) 2024-09-17T23:30:15.579 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:15.577+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:15.597 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.597 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.598 INFO:teuthology.orchestra.run.smithi139.stdout:3282 write 0xdb72c00 thru 0xdb77cb2 (0x50b3 bytes) 2024-09-17T23:30:15.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:15.610 INFO:teuthology.orchestra.run.smithi139.stdout:3289 trunc from 0xdb77e00 to 0x41bc000 2024-09-17T23:30:15.655 INFO:teuthology.orchestra.run.smithi139.stdout:3290 write 0x56ad400 thru 0x56b6db5 (0x99b6 bytes) 2024-09-17T23:30:15.656 INFO:teuthology.orchestra.run.smithi139.stdout:3293 read 0x4a31000 thru 0x4a33bd8 (0x2bd9 bytes) 2024-09-17T23:30:15.656 INFO:teuthology.orchestra.run.smithi139.stdout:3294 punch from 0x2785800 to 0x2791000, (0xb800 bytes) 2024-09-17T23:30:15.657 INFO:teuthology.orchestra.run.smithi139.stdout:3295 punch from 0x1876000 to 0x1882600, (0xc600 bytes) 2024-09-17T23:30:15.680 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:15.678+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:15.696 INFO:teuthology.orchestra.run.smithi139.stdout:3298 write 0x616d800 thru 0x6175811 (0x8012 bytes) 2024-09-17T23:30:15.697 INFO:teuthology.orchestra.run.smithi139.stdout:3299 write 0x5278200 thru 0x52841d3 (0xbfd4 bytes) 2024-09-17T23:30:15.697 INFO:teuthology.orchestra.run.smithi139.stdout:3301 clone 24 order 22 su 32768 sc 12 2024-09-17T23:30:15.737 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.737 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:15.790 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:15.786+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:15.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:15.896 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:15.896 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:15.897 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:15.895+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:15.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:15.983 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:15.981+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:16.000 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:16.000 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:16.017 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:16.018 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:16.034 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:16.034 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:16.051 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:16.051 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:16.058 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:16.058 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9550 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 2531328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.765085658, 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056192745000000002, 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9550 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290748+0000", 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0800000000000005e-07 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290751+0000", 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290757+0000", 2024-09-17T23:30:16.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4169999999999997e-06 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290762+0000", 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346584+0000", 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055821636000000001 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346864+0000", 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028079899999999998 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346941+0000", 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6173000000000004e-05 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9552 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 2539520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:16.063 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764848743999998, 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056146769999999999, 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9552 2024-09-17T23:30:16.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290985+0000", 2024-09-17T23:30:16.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8999999999999996e-07 2024-09-17T23:30:16.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.290994+0000", 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3399999999999998e-06 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291001+0000", 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1910000000000003e-06 2024-09-17T23:30:16.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291007+0000", 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8100000000000003e-06 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.346974+0000", 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055966828000000003 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347059+0000", 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5753000000000004e-05 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347131+0000", 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2157999999999996e-05 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9604 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3235840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764779370999999, 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05626722, 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9604 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291054+0000", 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291055+0000", 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291057+0000", 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7020000000000001e-06 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291064+0000", 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8260000000000004e-06 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291070+0000", 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:30:16.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347164+0000", 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056094887000000003 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347250+0000", 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5552999999999999e-05 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347321+0000", 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1310999999999995e-05 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9606 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764513679, 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05619619, 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9606 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291320+0000", 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1900000000000002e-07 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291324+0000", 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6619999999999998e-06 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291333+0000", 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5110000000000003e-06 2024-09-17T23:30:16.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291339+0000", 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.477e-06 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347352+0000", 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056012987 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347434+0000", 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1569000000000005e-05 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347516+0000", 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2365000000000002e-05 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9608 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3244032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764483517999999, 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056951664999999999, 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9608 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291350+0000", 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291351+0000", 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8299999999999997e-07 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291353+0000", 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.396e-06 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291360+0000", 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.199e-06 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291367+0000", 2024-09-17T23:30:16.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2230000000000001e-06 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.347546+0000", 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056178616000000001 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348232+0000", 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068600200000000001 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348302+0000", 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9646000000000004e-05 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9610 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764320827999999, 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056957865000000003, 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9610 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291513+0000", 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291517+0000", 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.393e-06 2024-09-17T23:30:16.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291526+0000", 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0129999999999999e-06 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291532+0000", 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8430000000000004e-06 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348331+0000", 2024-09-17T23:30:16.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056798279 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348405+0000", 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4418000000000005e-05 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348470+0000", 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5355e-05 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9613 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764250784000001, 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057056663000000001, 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9613 2024-09-17T23:30:16.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291583+0000", 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291585+0000", 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291593+0000", 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6170000000000001e-06 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291596+0000", 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.269e-06 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348498+0000", 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056902369000000001 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348573+0000", 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4962000000000005e-05 2024-09-17T23:30:16.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348639+0000", 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5874e-05 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9615 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.764087065999998, 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057044395999999997, 2024-09-17T23:30:16.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9615 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291746+0000", 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291747+0000", 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:30:16.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291749+0000", 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291754+0000", 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4410000000000003e-06 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291757+0000", 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4520000000000002e-06 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348665+0000", 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056908513000000001 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348732+0000", 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6180999999999995e-05 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348791+0000", 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9129000000000003e-05 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9620 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.763932712999999, 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057063968999999999, 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9620 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291901+0000", 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291909+0000", 2024-09-17T23:30:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5610000000000001e-06 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291916+0000", 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0679999999999999e-06 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291923+0000", 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3300000000000001e-06 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348817+0000", 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056893959000000001 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348898+0000", 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0377999999999995e-05 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.348965+0000", 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7146999999999999e-05 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9622 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3268608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.763874238, 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057231199000000003, 2024-09-17T23:30:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9622 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291959+0000", 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291960+0000", 2024-09-17T23:30:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291962+0000", 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7939999999999998e-06 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291970+0000", 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5009999999999998e-06 2024-09-17T23:30:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.291975+0000", 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999999e-06 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349015+0000", 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057039443000000002 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349104+0000", 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9791999999999997e-05 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.349190+0000", 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6055000000000002e-05 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9716 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.759728338999999, 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056085545000000001, 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9716 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296105+0000", 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296106+0000", 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296108+0000", 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.334e-06 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296114+0000", 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7830000000000004e-06 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296119+0000", 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352074+0000", 2024-09-17T23:30:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055955619999999998 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352121+0000", 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6857999999999998e-05 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352191+0000", 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9639000000000002e-05 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9722 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.759669421000002, 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056209310999999998, 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9722 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296164+0000", 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296362+0000", 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019768800000000001 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296366+0000", 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9419999999999997e-06 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296375+0000", 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6309999999999998e-06 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296392+0000", 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6758000000000001e-05 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352221+0000", 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055828950000000002 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352299+0000", 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8375999999999996e-05 2024-09-17T23:30:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352373+0000", 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3966000000000003e-05 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9723 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.759430513000002, 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056145054999999999, 2024-09-17T23:30:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9723 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296403+0000", 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296404+0000", 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8800000000000002e-07 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296411+0000", 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0809999999999996e-06 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296419+0000", 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0129999999999993e-06 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296424+0000", 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5879999999999997e-06 2024-09-17T23:30:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352405+0000", 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055980769 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352479+0000", 2024-09-17T23:30:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4018999999999997e-05 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352548+0000", 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8997000000000004e-05 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9729 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.759375332000001, 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056272003000000001, 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9729 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296458+0000", 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296459+0000", 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2799999999999996e-07 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296461+0000", 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4439999999999998e-06 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296468+0000", 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296472+0000", 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352583+0000", 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056111017999999999 2024-09-17T23:30:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352665+0000", 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1148000000000003e-05 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352730+0000", 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5477999999999999e-05 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9731 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3973120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.759353947000001, 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056506531999999998, 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9731 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.296480+0000", 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297569+0000", 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001089394 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297574+0000", 2024-09-17T23:30:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7260000000000002e-06 2024-09-17T23:30:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297583+0000", 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4169999999999998e-06 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297590+0000", 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9099999999999999e-06 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352775+0000", 2024-09-17T23:30:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055185161000000003 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352867+0000", 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2090999999999996e-05 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.352986+0000", 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011883300000000001 2024-09-17T23:30:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9743 3.13 3:cc79abe2:::rbd_data.11521dab6f51.0000000000000000:11 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.758084332999999, 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055972380000000002, 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9743 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297749+0000", 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297750+0000", 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297752+0000", 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297760+0000", 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1839999999999999e-06 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297766+0000", 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353551+0000", 2024-09-17T23:30:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055784783999999997 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353645+0000", 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3760999999999999e-05 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353721+0000", 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6907000000000001e-05 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9744 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 3989504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.758059464999999, 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056155391999999998, 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9744 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297774+0000", 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297775+0000", 2024-09-17T23:30:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.51e-07 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.297777+0000", 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5639999999999999e-06 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298431+0000", 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065376599999999996 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.298441+0000", 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8709999999999992e-06 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353773+0000", 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.055332332999999997 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353856+0000", 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3122999999999997e-05 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.353929+0000", 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3183999999999996e-05 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:9795 3.13 3:c92b85a4:::rbd_data.11521dab6f51.0000000000000001:11 [read 4677632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.75567564, 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056138625999999997, 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 9795 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.300158+0000", 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301567+0000", 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001409492 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301573+0000", 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2159999999999997e-06 2024-09-17T23:30:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301582+0000", 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5619999999999993e-06 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.301588+0000", 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356166+0000", 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054577696000000002 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356253+0000", 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6526000000000005e-05 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.356296+0000", 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3918999999999997e-05 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12612 3.13 3:ccf171c8:::rbd_data.115262cdab6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.545142386999999, 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.063461453000000001, 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12612 2024-09-17T23:30:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510691+0000", 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.510693+0000", 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9e-06 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539114+0000", 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028420697000000002 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539145+0000", 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1488000000000001e-05 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539159+0000", 2024-09-17T23:30:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3737000000000001e-05 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539226+0000", 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7572999999999998e-05 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539337+0000", 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110849 2024-09-17T23:30:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.540857+0000", 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001520043 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.571640+0000", 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030782961000000001 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574080+0000", 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024394529999999998 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574120+0000", 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0459e-05 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.574153+0000", 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2292999999999997e-05 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12613 3.9 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.516644384999999, 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.13090919500000001, 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12613 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539189+0000", 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.539191+0000", 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.59e-06 2024-09-17T23:30:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.595998+0000", 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056807745999999999 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596021+0000", 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2574000000000001e-05 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596030+0000", 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1670000000000005e-06 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596150+0000", 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119688 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.596301+0000", 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015097400000000001 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598103+0000", 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001802281 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.610230+0000", 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012126468 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.669969+0000", 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059739318999999999 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670041+0000", 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1834000000000004e-05 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.670098+0000", 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7553999999999998e-05 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.169 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:16.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:16.171 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:16.171 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:16.172 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:16.172 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:16.173 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:30:16.174 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:16.168+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:16.230 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone23 from 0x6175a00 (overlap 0x1d66400) to 0x2f0a00 2024-09-17T23:30:16.271 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:16.269+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:16.372 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:16.370+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:16.392 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5837 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360629+0000", 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025341188000000001, 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025362638, 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5837 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360629+0000", 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360629+0000", 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360630+0000", 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360632+0000", 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5969999999999999e-06 2024-09-17T23:30:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360639+0000", 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.725e-06 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360644+0000", 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4399999999999996e-06 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5839 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360652+0000", 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025318692, 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025406808999999999, 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5839 2024-09-17T23:30:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360652+0000", 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360652+0000", 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360652+0000", 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360654+0000", 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360661+0000", 2024-09-17T23:30:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360666+0000", 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4650000000000002e-06 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5841 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360673+0000", 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025297118, 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025455518, 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5841 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360673+0000", 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360673+0000", 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360674+0000", 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360676+0000", 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1859999999999999e-06 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360682+0000", 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5039999999999999e-06 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360687+0000", 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1869999999999996e-06 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5971 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361358+0000", 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024611841999999998, 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024840655, 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5971 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361358+0000", 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361358+0000", 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361359+0000", 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2899999999999999e-07 2024-09-17T23:30:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361361+0000", 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.984e-06 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361367+0000", 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361372+0000", 2024-09-17T23:30:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5842 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360694+0000", 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025275988999999999, 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025561605000000001, 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5842 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360694+0000", 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360694+0000", 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360695+0000", 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2899999999999999e-07 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360697+0000", 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360704+0000", 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7619999999999998e-06 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360709+0000", 2024-09-17T23:30:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2410000000000003e-06 2024-09-17T23:30:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5972 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361379+0000", 2024-09-17T23:30:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024591117999999999, 2024-09-17T23:30:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024933845, 2024-09-17T23:30:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5972 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361379+0000", 2024-09-17T23:30:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361379+0000", 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361380+0000", 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361387+0000", 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6090000000000003e-06 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361479+0000", 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1974999999999999e-05 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361488+0000", 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7029999999999995e-06 2024-09-17T23:30:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5844 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360715+0000", 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025254819000000001, 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025654745, 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5844 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360715+0000", 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360715+0000", 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360716+0000", 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360718+0000", 2024-09-17T23:30:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0449999999999999e-06 2024-09-17T23:30:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360724+0000", 2024-09-17T23:30:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4330000000000001e-06 2024-09-17T23:30:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360730+0000", 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5589999999999996e-06 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5975 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361499+0000", 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024470918000000001, 2024-09-17T23:30:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024926699, 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5975 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361499+0000", 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361499+0000", 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361500+0000", 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7899999999999998e-07 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361504+0000", 2024-09-17T23:30:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5080000000000001e-06 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361513+0000", 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4830000000000005e-06 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361519+0000", 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7599999999999999e-06 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5846 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360737+0000", 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025233731999999998, 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025745117000000001, 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5846 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360737+0000", 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360737+0000", 2024-09-17T23:30:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360737+0000", 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360739+0000", 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.137e-06 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360745+0000", 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3629999999999999e-06 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360750+0000", 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0719999999999999e-06 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5976 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361526+0000", 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02444375, 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02501345, 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5976 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361526+0000", 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361526+0000", 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361527+0000", 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361529+0000", 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361542+0000", 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2855e-05 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361548+0000", 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5856 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360757+0000", 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025213250999999999, 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025848051, 2024-09-17T23:30:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5856 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360757+0000", 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360757+0000", 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360757+0000", 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360760+0000", 2024-09-17T23:30:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.142e-06 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360766+0000", 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.37e-06 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360771+0000", 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5980 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361556+0000", 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024413791000000001, 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025104197000000002, 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5980 2024-09-17T23:30:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361556+0000", 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361556+0000", 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361557+0000", 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361559+0000", 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361572+0000", 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2286999999999999e-05 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361578+0000", 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5858 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360777+0000", 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025192831999999998, 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025938372000000001, 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5858 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360777+0000", 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360777+0000", 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360778+0000", 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360780+0000", 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0700000000000001e-06 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360786+0000", 2024-09-17T23:30:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3910000000000003e-06 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360791+0000", 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5981 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361586+0000", 2024-09-17T23:30:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024384085999999999, 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025186713999999999, 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5981 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361586+0000", 2024-09-17T23:30:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361586+0000", 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361587+0000", 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7599999999999997e-07 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361589+0000", 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361887+0000", 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000297664 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361894+0000", 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7530000000000004e-06 2024-09-17T23:30:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5860 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360798+0000", 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025171763999999999, 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026030550999999999, 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5860 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360798+0000", 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360798+0000", 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360799+0000", 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360801+0000", 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0729999999999999e-06 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360808+0000", 2024-09-17T23:30:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7920000000000004e-06 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360813+0000", 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0980000000000001e-06 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5862 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360820+0000", 2024-09-17T23:30:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025150057999999999, 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026070317999999999, 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5862 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360820+0000", 2024-09-17T23:30:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360820+0000", 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360821+0000", 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360823+0000", 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2139999999999999e-06 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360830+0000", 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7490000000000001e-06 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360835+0000", 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5897 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360911+0000", 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025059595, 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026012364, 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5897 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360911+0000", 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360911+0000", 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360911+0000", 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360913+0000", 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360921+0000", 2024-09-17T23:30:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7800000000000001e-06 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360926+0000", 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.22e-06 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5898 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360933+0000", 2024-09-17T23:30:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025037660999999999, 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026021224999999999, 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5898 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360933+0000", 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360933+0000", 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360933+0000", 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2800000000000002e-07 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360935+0000", 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.092e-06 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360941+0000", 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3160000000000002e-06 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360946+0000", 2024-09-17T23:30:16.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0649999999999998e-06 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6011 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.375893+0000", 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010077696000000001, 2024-09-17T23:30:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011089827999999999, 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6011 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375893+0000", 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375893+0000", 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375893+0000", 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375897+0000", 2024-09-17T23:30:16.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-06 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375907+0000", 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8239999999999995e-06 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375937+0000", 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9719000000000001e-05 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5902 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360953+0000", 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025016914000000001, 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026055967999999999, 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5902 2024-09-17T23:30:16.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360953+0000", 2024-09-17T23:30:16.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360953+0000", 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360954+0000", 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360956+0000", 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0379999999999998e-06 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360962+0000", 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:30:16.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360967+0000", 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9300000000000002e-06 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6014 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.375949+0000", 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010020973000000001, 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011086745, 2024-09-17T23:30:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6014 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375949+0000", 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375949+0000", 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375950+0000", 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7800000000000001e-07 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375953+0000", 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4910000000000001e-06 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375961+0000", 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6839999999999996e-06 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375968+0000", 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7150000000000001e-06 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5903 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360974+0000", 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024996375000000001, 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026089950000000001, 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5903 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360974+0000", 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360974+0000", 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360974+0000", 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5999999999999999e-07 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360977+0000", 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.492e-06 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360983+0000", 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.562e-06 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360988+0000", 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7539999999999997e-06 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6017 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.375984+0000", 2024-09-17T23:30:16.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0099863910000000007, 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011109846, 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6017 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375984+0000", 2024-09-17T23:30:16.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375984+0000", 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375984+0000", 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375987+0000", 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9289999999999998e-06 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.375996+0000", 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376002+0000", 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6489999999999996e-06 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5783 3.14 3:2b831442:::rbd_data.1152cba0bdb.000000000000000b:19 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.359781+0000", 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026188955999999999, 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027372027, 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5783 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359781+0000", 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359781+0000", 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359782+0000", 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3200000000000005e-07 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359784+0000", 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1440000000000001e-06 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359791+0000", 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:16.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359795+0000", 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.387043+0000", 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027247304999999999 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.387132+0000", 2024-09-17T23:30:16.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9015000000000001e-05 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5905 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360995+0000", 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024975174999999999, 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026227059, 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5905 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360995+0000", 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360995+0000", 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360995+0000", 2024-09-17T23:30:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-17T23:30:16.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360998+0000", 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3329999999999999e-06 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361005+0000", 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361010+0000", 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6019 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376010+0000", 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0099604930000000008, 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011268281999999999, 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6019 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376010+0000", 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376010+0000", 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376010+0000", 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376012+0000", 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376019+0000", 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3990000000000001e-06 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376024+0000", 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0019999999999997e-06 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5785 3.14 3:2b831442:::rbd_data.1152cba0bdb.000000000000000b:19 [read 1277952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.359802+0000", 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026167956999999999, 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027537874, 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5785 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359802+0000", 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359802+0000", 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359803+0000", 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359805+0000", 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1210000000000001e-06 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359812+0000", 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359817+0000", 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-06 2024-09-17T23:30:16.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.387295+0000", 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027477926 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5907 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361016+0000", 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024953767000000002, 2024-09-17T23:30:16.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026385522000000002, 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5907 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361016+0000", 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361016+0000", 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361017+0000", 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361019+0000", 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361026+0000", 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5889999999999999e-06 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361031+0000", 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0669999999999999e-06 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5789 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.359824+0000", 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026146578, 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027633615, 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5789 2024-09-17T23:30:16.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359824+0000", 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359824+0000", 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359824+0000", 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359830+0000", 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3749999999999999e-06 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359837+0000", 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8329999999999996e-06 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359843+0000", 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3109999999999996e-06 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5910 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361038+0000", 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024932622000000002, 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026475136999999999, 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5910 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361038+0000", 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361038+0000", 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361038+0000", 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.44e-07 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361040+0000", 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1859999999999999e-06 2024-09-17T23:30:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361047+0000", 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.567e-06 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361052+0000", 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0490000000000003e-06 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5791 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.359850+0000", 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02611987, 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027731424000000001, 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5791 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359850+0000", 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359850+0000", 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359851+0000", 2024-09-17T23:30:16.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359853+0000", 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359860+0000", 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5590000000000002e-06 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359865+0000", 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5911 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361058+0000", 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024911843999999999, 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026584726999999999, 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5911 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361058+0000", 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361058+0000", 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361059+0000", 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361061+0000", 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.114e-06 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361067+0000", 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4139999999999999e-06 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361072+0000", 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8420000000000004e-06 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6025 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376074+0000", 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098958450000000003, 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011628526, 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6025 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376074+0000", 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376074+0000", 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376075+0000", 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376077+0000", 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.092e-06 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376082+0000", 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.57e-06 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376087+0000", 2024-09-17T23:30:16.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0010000000000001e-06 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5799 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.359871+0000", 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026098778999999999, 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027889872, 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5799 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359871+0000", 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359871+0000", 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359872+0000", 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2399999999999999e-07 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359874+0000", 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1859999999999999e-06 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359881+0000", 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4840000000000001e-06 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359886+0000", 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0950000000000003e-06 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5912 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361079+0000", 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024891475, 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0267832, 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5912 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361079+0000", 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361079+0000", 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361079+0000", 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361081+0000", 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9649999999999998e-06 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361087+0000", 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3709999999999996e-06 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361093+0000", 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1240000000000004e-06 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6030 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376094+0000", 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098765550000000004, 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011824935999999999, 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6030 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376094+0000", 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376094+0000", 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376094+0000", 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376102+0000", 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5140000000000004e-06 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376108+0000", 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2999999999999998e-06 2024-09-17T23:30:16.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376112+0000", 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5889999999999996e-06 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5802 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.359892+0000", 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.026078480000000001, 2024-09-17T23:30:16.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028074310000000002, 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5802 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359892+0000", 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359892+0000", 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359892+0000", 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359894+0000", 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9769999999999999e-06 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359901+0000", 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.359906+0000", 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1830000000000002e-06 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5913 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361099+0000", 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024870976, 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026895063, 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5913 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.531 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361099+0000", 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361099+0000", 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361100+0000", 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2800000000000002e-07 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361102+0000", 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361108+0000", 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361113+0000", 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9910000000000002e-06 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6032 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376118+0000", 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098518119999999997, 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011904962999999999, 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6032 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376118+0000", 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376118+0000", 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376119+0000", 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376121+0000", 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9230000000000001e-06 2024-09-17T23:30:16.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376126+0000", 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5300000000000004e-06 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376131+0000", 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5390000000000001e-06 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5921 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361120+0000", 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024850385999999999, 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026958626999999999, 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5921 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361120+0000", 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361120+0000", 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361120+0000", 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.53e-07 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361127+0000", 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5470000000000002e-06 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361134+0000", 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9580000000000001e-06 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361139+0000", 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6033 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376753+0000", 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0092171730000000004, 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011384368000000001, 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6033 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376753+0000", 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376753+0000", 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376754+0000", 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2900000000000003e-07 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376758+0000", 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376770+0000", 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2097e-05 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376778+0000", 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2119999999999997e-06 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6036 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376790+0000", 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091805819999999996, 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011404577000000001, 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6036 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376790+0000", 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376790+0000", 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376790+0000", 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7299999999999996e-07 2024-09-17T23:30:16.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376793+0000", 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6220000000000001e-06 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376800+0000", 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6290000000000001e-06 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376807+0000", 2024-09-17T23:30:16.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5440000000000004e-06 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5813 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360453+0000", 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025517284000000001, 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027797584, 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5813 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360453+0000", 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360453+0000", 2024-09-17T23:30:16.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360453+0000", 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360456+0000", 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.644e-06 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360464+0000", 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4499999999999998e-06 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360469+0000", 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7749999999999998e-06 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6042 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376816+0000", 2024-09-17T23:30:16.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091545080000000004, 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011495823, 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6042 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376816+0000", 2024-09-17T23:30:16.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376816+0000", 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376816+0000", 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376819+0000", 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6790000000000001e-06 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376826+0000", 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.905e-06 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376831+0000", 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5815 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360478+0000", 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025492529, 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027889972999999998, 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5815 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360478+0000", 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360478+0000", 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360478+0000", 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6499999999999999e-07 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360481+0000", 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360487+0000", 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9149999999999999e-06 2024-09-17T23:30:16.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360493+0000", 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6380000000000001e-06 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5949 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361189+0000", 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024781383000000001, 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027242717999999999, 2024-09-17T23:30:16.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5949 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361189+0000", 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361189+0000", 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361189+0000", 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361192+0000", 2024-09-17T23:30:16.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2539999999999999e-06 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361199+0000", 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0670000000000003e-06 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361204+0000", 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5818 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360500+0000", 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025470278999999998, 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027988064, 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5818 2024-09-17T23:30:16.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360500+0000", 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360500+0000", 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360500+0000", 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360503+0000", 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1050000000000002e-06 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360509+0000", 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7259999999999996e-06 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360515+0000", 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3199999999999999e-06 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5950 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361210+0000", 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024759758999999999, 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027337429999999999, 2024-09-17T23:30:16.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5950 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361210+0000", 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361210+0000", 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361211+0000", 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3200000000000005e-07 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361213+0000", 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1880000000000001e-06 2024-09-17T23:30:16.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361220+0000", 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5540000000000003e-06 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361225+0000", 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0359999999999997e-06 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6043 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376839+0000", 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091317299999999994, 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011770843, 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6043 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376839+0000", 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376839+0000", 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376839+0000", 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376842+0000", 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5280000000000002e-06 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376848+0000", 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.72e-06 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376854+0000", 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3709999999999999e-06 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5820 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360521+0000", 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02544898, 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028148787000000002, 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5820 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360521+0000", 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360521+0000", 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360522+0000", 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5400000000000002e-07 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360524+0000", 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1509999999999998e-06 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360532+0000", 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8590000000000005e-06 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360537+0000", 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4049999999999999e-06 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5955 3.14 3.4228c1d4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361232+0000", 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024738705, 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027494665000000001, 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5955 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361232+0000", 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361232+0000", 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361232+0000", 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9400000000000001e-07 2024-09-17T23:30:16.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361234+0000", 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1270000000000001e-06 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361241+0000", 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6780000000000003e-06 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361246+0000", 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:6044 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.376862+0000", 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091082929999999999, 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011923935, 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6044 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376862+0000", 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376862+0000", 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376862+0000", 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376865+0000", 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4710000000000001e-06 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376872+0000", 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0029999999999997e-06 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.376877+0000", 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1649999999999997e-06 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5962 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361252+0000", 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024717857999999999, 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027588067000000001, 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5962 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361252+0000", 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361252+0000", 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361253+0000", 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9299999999999998e-07 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361255+0000", 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361261+0000", 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3659999999999997e-06 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361267+0000", 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1999999999999999e-06 2024-09-17T23:30:16.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5964 3.14 3.42aafcb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.361275+0000", 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.024695516000000001, 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027626554000000001, 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5964 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361275+0000", 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361275+0000", 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361275+0000", 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361278+0000", 2024-09-17T23:30:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.666e-06 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361285+0000", 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6830000000000002e-06 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.361290+0000", 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1580000000000004e-06 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:5836 3.14 3.466c9fb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.360599+0000", 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.025371112000000001, 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028357949, 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 5836 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360599+0000", 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360599+0000", 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360600+0000", 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5699999999999998e-07 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360608+0000", 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6149999999999997e-06 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360615+0000", 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.360621+0000", 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3639999999999998e-06 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 55 2024-09-17T23:30:16.593 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:16.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:16.595 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:16.593+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:16.695 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:16.694+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:16.745 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:16.745 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:16.745 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:16.745 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:16.745 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:16.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:16.796 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:16.794+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:16.897 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:16.896+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:16.909 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.435376701999999, 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.577540889000002, 2024-09-17T23:30:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.576848596000001, 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.576837014999999, 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.576309768000002, 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.576275785, 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.575965193000002, 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.575939059, 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.575441152, 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.574840062, 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.574291426999999, 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.574083828999999, 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.564467927999999, 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.564457694000001, 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.564447826999999, 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.564437679000001, 2024-09-17T23:30:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.563967811000001, 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.563945695000001, 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.563658, 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.378271829999999, 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.006 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:17.007 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T23:30:17.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:17.009 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:17.006+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:17.014 INFO:teuthology.orchestra.run.smithi139.stdout:3302 read 0x27a4600 thru 0x27a7af3 (0x34f4 bytes) 2024-09-17T23:30:17.015 INFO:teuthology.orchestra.run.smithi139.stdout:3303 punch from 0x5188000 to 0x5194a00, (0xca00 bytes) 2024-09-17T23:30:17.032 INFO:teuthology.orchestra.run.smithi139.stdout:3304 read 0x32c2000 thru 0x32c2417 (0x418 bytes) 2024-09-17T23:30:17.039 INFO:teuthology.orchestra.run.smithi139.stdout:3305 trunc from 0x6175a00 to 0x6a8200 2024-09-17T23:30:17.109 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:17.107+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:17.131 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:17.131 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:17.131 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 3, 4, 5, 2] 2024-09-17T23:30:17.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T23:30:17.210 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:17.208+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:17.227 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:17.228 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:17.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:17.277 INFO:teuthology.orchestra.run.smithi139.stdout:3306 trunc from 0x6a8200 to 0x8a1a800 2024-09-17T23:30:17.292 INFO:teuthology.orchestra.run.smithi139.stdout:3310 write 0x514be00 thru 0x515b062 (0xf263 bytes) 2024-09-17T23:30:17.293 INFO:teuthology.orchestra.run.smithi139.stdout:3311 write 0x67db200 thru 0x67dea05 (0x3806 bytes) 2024-09-17T23:30:17.293 INFO:teuthology.orchestra.run.smithi139.stdout:3317 trunc from 0x8a1a800 to 0x9860e00 2024-09-17T23:30:17.312 INFO:teuthology.orchestra.run.smithi139.stdout:3318 write 0x2d1a400 thru 0x2d236f7 (0x92f8 bytes) 2024-09-17T23:30:17.320 INFO:teuthology.orchestra.run.smithi139.stdout:3321 write 0x90cb200 thru 0x90d26d4 (0x74d5 bytes) 2024-09-17T23:30:17.320 INFO:teuthology.orchestra.run.smithi139.stdout:3322 write 0x8f67a00 thru 0x8f74694 (0xcc95 bytes) 2024-09-17T23:30:17.321 INFO:teuthology.orchestra.run.smithi139.stdout:3323 trunc from 0x9860e00 to 0xad75a00 2024-09-17T23:30:17.325 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:17.324+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:17.342 INFO:teuthology.orchestra.run.smithi139.stdout:3324 read 0xa6cb400 thru 0xa6dafb2 (0xfbb3 bytes) 2024-09-17T23:30:17.350 INFO:teuthology.orchestra.run.smithi139.stdout:3325 punch from 0x24e6000 to 0x24e9800, (0x3800 bytes) 2024-09-17T23:30:17.351 INFO:teuthology.orchestra.run.smithi139.stdout:3326 read 0x3a56200 thru 0x3a615e8 (0xb3e9 bytes) 2024-09-17T23:30:17.351 INFO:teuthology.orchestra.run.smithi139.stdout:3332 write 0x7f51800 thru 0x7f60da6 (0xf5a7 bytes) 2024-09-17T23:30:17.351 INFO:teuthology.orchestra.run.smithi139.stdout:3333 punch from 0x91a4a00 to 0x91a7800, (0x2e00 bytes) 2024-09-17T23:30:17.351 INFO:teuthology.orchestra.run.smithi139.stdout:3335 write 0x9922000 thru 0x992c255 (0xa256 bytes) 2024-09-17T23:30:17.351 INFO:teuthology.orchestra.run.smithi139.stdout:3337 write 0xbfa200 thru 0xc046d2 (0xa4d3 bytes) 2024-09-17T23:30:17.363 INFO:teuthology.orchestra.run.smithi139.stdout:3338 trunc from 0xad75a00 to 0x3b8a600 2024-09-17T23:30:17.384 INFO:teuthology.orchestra.run.smithi139.stdout:3343 read 0xfa1600 thru 0xfaf0ae (0xdaaf bytes) 2024-09-17T23:30:17.385 INFO:teuthology.orchestra.run.smithi139.stdout:3344 read 0x336c000 thru 0x336cedf (0xee0 bytes) 2024-09-17T23:30:17.386 INFO:teuthology.orchestra.run.smithi139.stdout:3345 punch from 0xfa5800 to 0xfaee00, (0x9600 bytes) 2024-09-17T23:30:17.386 INFO:teuthology.orchestra.run.smithi139.stdout:3346 write 0x25f1600 thru 0x2600284 (0xec85 bytes) 2024-09-17T23:30:17.386 INFO:teuthology.orchestra.run.smithi139.stdout:3347 read 0x70be00 thru 0x7173b8 (0xb5b9 bytes) 2024-09-17T23:30:17.398 INFO:teuthology.orchestra.run.smithi139.stdout:3348 trunc from 0x3b8a600 to 0xc188a00 2024-09-17T23:30:17.408 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:17.408 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:17.408 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:17.408 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:17.408 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:17.409 INFO:teuthology.orchestra.run.smithi139.stdout:3350 read 0x64e000 thru 0x650af4 (0x2af5 bytes) 2024-09-17T23:30:17.412 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:17.410+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:17.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:17.417 INFO:teuthology.orchestra.run.smithi139.stdout:3351 trunc from 0xc188a00 to 0x6962800 2024-09-17T23:30:17.443 INFO:teuthology.orchestra.run.smithi139.stdout:3352 write 0xe647e00 thru 0xe653fe2 (0xc1e3 bytes) 2024-09-17T23:30:17.459 INFO:teuthology.orchestra.run.smithi139.stdout:3353 trunc from 0xe654000 to 0xb2b200 2024-09-17T23:30:17.504 INFO:teuthology.orchestra.run.smithi139.stdout:3354 trunc from 0xb2b200 to 0x9464800 2024-09-17T23:30:17.513 INFO:teuthology.orchestra.run.smithi139.stdout:3356 read 0x5eba00 thru 0x5fabea (0xf1eb bytes) 2024-09-17T23:30:17.514 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:17.511+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:17.514 INFO:teuthology.orchestra.run.smithi139.stdout:3357 write 0x2b50c00 thru 0x2b53e41 (0x3242 bytes) 2024-09-17T23:30:17.515 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:30:17.513+0000 7f21c94c5640 -1 mon.a@0(leader).osd e65 definitely_dead 0 2024-09-17T23:30:17.515 INFO:teuthology.orchestra.run.smithi139.stdout:3358 punch from 0x3216e00 to 0x3218600, (0x1800 bytes) 2024-09-17T23:30:17.516 INFO:teuthology.orchestra.run.smithi139.stdout:3359 trunc from 0x9464800 to 0xa349200 2024-09-17T23:30:17.534 INFO:teuthology.orchestra.run.smithi139.stdout:3360 write 0x1eca800 thru 0x1ed69ae (0xc1af bytes) 2024-09-17T23:30:17.534 INFO:teuthology.orchestra.run.smithi139.stdout:3365 punch from 0x116f000 to 0x116fe00, (0xe00 bytes) 2024-09-17T23:30:17.534 INFO:teuthology.orchestra.run.smithi139.stdout:3366 write 0x1e04400 thru 0x1e084b1 (0x40b2 bytes) 2024-09-17T23:30:17.536 INFO:teuthology.orchestra.run.smithi139.stdout:3367 read 0x7eff200 thru 0x7f0bc88 (0xca89 bytes) 2024-09-17T23:30:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:3369 punch from 0x7bedc00 to 0x7bf1a00, (0x3e00 bytes) 2024-09-17T23:30:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:3373 read 0x6724400 thru 0x672a7b1 (0x63b2 bytes) 2024-09-17T23:30:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:3374 read 0x7c37000 thru 0x7c3845c (0x145d bytes) 2024-09-17T23:30:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:3377 write 0x781200 thru 0x790eb0 (0xfcb1 bytes) 2024-09-17T23:30:17.537 INFO:teuthology.orchestra.run.smithi139.stdout:3378 read 0x4a1d400 thru 0x4a2357d (0x617e bytes) 2024-09-17T23:30:17.542 INFO:teuthology.orchestra.run.smithi139.stdout:3381 trunc from 0xa349200 to 0xe6e2000 2024-09-17T23:30:17.562 INFO:teuthology.orchestra.run.smithi139.stdout:3383 trunc from 0xe6e2000 to 0xab15c00 2024-09-17T23:30:17.571 INFO:teuthology.orchestra.run.smithi139.stdout:3384 read 0x795c000 thru 0x7968385 (0xc386 bytes) 2024-09-17T23:30:17.572 INFO:teuthology.orchestra.run.smithi139.stdout:3385 write 0x75f8e00 thru 0x7606441 (0xd642 bytes) 2024-09-17T23:30:17.572 INFO:teuthology.orchestra.run.smithi139.stdout:3387 flatten 2024-09-17T23:30:17.582 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.543064203999997, 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.542891263999998, 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.542851394000003, 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.248347166999999, 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.000645618, 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.990713352, 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.977486302999999, 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436957147000001, 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436935757000001, 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436918628000001, 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436901345999999, 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436884317000001, 2024-09-17T23:30:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436851442999998, 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.436263737000001, 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.414471596, 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.413934407999999, 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.413448292, 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:17.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.34130569, 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:17.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:17.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3412838790000001, 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:17.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3318410090000001, 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:17.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:17.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:17.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:17.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:17.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:17.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:17.703 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:17.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:17.705 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:17.702+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3388 write 0x463ca00 thru 0x4643262 (0x6863 bytes) 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3389 write 0xed9f000 thru 0xedac81d (0xd81e bytes) 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3390 write 0xdff9a00 thru 0xe006799 (0xcd9a bytes) 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3391 trunc from 0xedaca00 to 0x39eaa00 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3392 write 0x154f800 thru 0x1554ffe (0x57ff bytes) 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3393 write 0xadeda00 thru 0xadf716d (0x976e bytes) 2024-09-17T23:30:17.706 INFO:teuthology.orchestra.run.smithi139.stdout:3394 trunc from 0xadf7200 to 0xd9bf800 2024-09-17T23:30:17.707 INFO:teuthology.orchestra.run.smithi139.stdout:3397 write 0x7c2e000 thru 0x7c2e684 (0x685 bytes) 2024-09-17T23:30:17.707 INFO:teuthology.orchestra.run.smithi139.stdout:3400 write 0xa499600 thru 0xa4a6516 (0xcf17 bytes) 2024-09-17T23:30:17.707 INFO:teuthology.orchestra.run.smithi139.stdout:3401 write 0x8fc5200 thru 0x8fd34ff (0xe300 bytes) 2024-09-17T23:30:17.707 INFO:teuthology.orchestra.run.smithi139.stdout:3403 write 0x526200 thru 0x52adce (0x4bcf bytes) 2024-09-17T23:30:17.707 INFO:teuthology.orchestra.run.smithi139.stdout:3404 punch from 0xa3bfc00 to 0xa3c5600, (0x5a00 bytes) 2024-09-17T23:30:17.707 INFO:teuthology.orchestra.run.smithi139.stdout:3405 clone 25 order 23 su 2097152 sc 13 2024-09-17T23:30:17.805 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:17.803+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:17.814 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:17.814 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:17.814 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:17.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:17.907 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:17.905+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:17.970 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:17.971 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:17.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:18.020 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:18.018+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:18.108 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:18.106+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:18.128 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:18.128 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.250497221000003, 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:18.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.135 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234872091, 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:18.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:18.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234834435000003, 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:18.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234817004, 2024-09-17T23:30:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:18.150 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:18.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:18.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:18.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:18.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:18.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:18.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234782072999998, 2024-09-17T23:30:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:18.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:18.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:18.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:18.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.174 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234750652999999, 2024-09-17T23:30:18.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:18.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:18.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:18.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:18.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234734027000002, 2024-09-17T23:30:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:18.188 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:18.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:18.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:18.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:18.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.201 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234712332999997, 2024-09-17T23:30:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.203 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:18.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:18.214 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.234700615000001, 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:18.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.219 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.670498272000003, 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:18.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.670475805999999, 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:30:18.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:30:18.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.653137522000002, 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:18.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:30:18.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:30:18.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.651229628000003, 2024-09-17T23:30:18.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:30:18.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:30:18.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.649244557999999, 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.238 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:30:18.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:30:18.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.649234962999998, 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:18.242 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:18.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12611 3.18 3:1c8edc2b:::rbd_data.115262cdab6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:18.246 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.615924242999998, 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.070059079999999996, 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12611 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:18.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509321+0000", 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-06 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553867+0000", 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044545920000000003 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553893+0000", 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6094000000000001e-05 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553906+0000", 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3079e-05 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553972+0000", 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6630000000000004e-05 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.554090+0000", 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117364 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.555383+0000", 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001293188 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.577545+0000", 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.022161943 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579298+0000", 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017536349999999999 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579346+0000", 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7858000000000002e-05 2024-09-17T23:30:18.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579378+0000", 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2011000000000002e-05 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12614 3.2 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.571303460999999, 2024-09-17T23:30:18.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.100611516, 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12614 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553942+0000", 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.477e-06 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599428+0000", 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045486208 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599454+0000", 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6551999999999999e-05 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599467+0000", 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2339e-05 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599548+0000", 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599660+0000", 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011209699999999999 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.601086+0000", 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014260550000000001 2024-09-17T23:30:18.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.612776+0000", 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011690010000000001 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654410+0000", 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.041634517000000003 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654486+0000", 2024-09-17T23:30:18.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6260999999999994e-05 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654552+0000", 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5086999999999995e-05 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4968.0:12613 3.9 e48/19 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head v 48'120, mlcod=45'117)", 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.527045366999999, 2024-09-17T23:30:18.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049712148999999997, 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598199+0000", 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-07 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637181+0000", 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.038982528000000002 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637209+0000", 2024-09-17T23:30:18.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7317e-05 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637219+0000", 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0372e-05 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637307+0000", 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7867000000000004e-05 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637345+0000", 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7735999999999999e-05 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647845+0000", 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010500012 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647910+0000", 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5662999999999998e-05 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.262 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12676 3.1b 3:d8a99671:::rbd_data.115262cdab6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.520800753, 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.074765842999999998, 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12676 2024-09-17T23:30:18.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604445+0000", 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1399999999999998e-06 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638570+0000", 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.034124814000000003 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638586+0000", 2024-09-17T23:30:18.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6460000000000002e-05 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638611+0000", 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5216e-05 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638676+0000", 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5048999999999993e-05 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638792+0000", 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115233 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.640186+0000", 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013940090000000001 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657756+0000", 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017570240000000001 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679079+0000", 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.021323312000000001 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679150+0000", 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1086000000000003e-05 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679209+0000", 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8284e-05 2024-09-17T23:30:18.268 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5039.0:2696 3.b 3:d04d06ca:::rbd_data.1152adcf8b1e.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 589824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:17.148959+0000", 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.97628442500000001, 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.056869014000000002, 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5039", 2024-09-17T23:30:18.269 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1181598355", 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2696 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148959+0000", 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148959+0000", 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148960+0000", 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6400000000000001e-07 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148964+0000", 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4839999999999998e-06 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148974+0000", 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5249999999999994e-06 2024-09-17T23:30:18.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148981+0000", 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6889999999999998e-06 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.149034+0000", 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3451e-05 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:18.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.149115+0000", 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1118000000000005e-05 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.149213+0000", 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8055999999999998e-05 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.152526+0000", 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033133300000000002 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:18.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.205735+0000", 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.053208593999999998 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.205779+0000", 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4163999999999999e-05 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.205828+0000", 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8838999999999998e-05 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.274 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:18.275 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:18.275 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:18.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:18.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:30:18.278 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.2. 2024-09-17T23:30:18.278 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:18.278 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:18.275+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:18.378 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:18.376+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:18.430 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:18.430 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:18.430 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:18.438 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:18.456 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:18.456 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:18.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:18.474 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:18.482 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:18.480+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:18.507 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:18.507 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:18.525 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:18.525 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:18.544 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:18.545 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:18.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:18.556 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:18.557 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:18.573 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:18.574 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:18.594 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:18.592+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:18.681 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:18.679+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:18.700 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:18.701 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:18.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:18.781 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:18.779+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:18.815 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:18.816 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:18.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:18.882 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:18.881+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:18.970 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:18.970 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:18.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:19.023 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:19.021+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:19.084 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:19.082+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:19.087 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.087 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:19.153 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:30:19.185 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:19.183+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:19.218 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.218 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:19.286 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:19.284+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:19.376 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.376 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:19.386 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:19.384+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:19.487 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:19.485+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:19.492 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.492 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:19.588 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:19.587+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:19.648 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.649 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:19.690 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:19.688+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:19.790 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:19.789+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:19.804 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.804 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:19.892 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:19.890+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:19.960 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:19.960 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:19.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:20.015 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:20.013+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:20.077 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:20.077 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:20.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:20.128 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:20.126+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:20.194 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:20.192+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:20.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:20.296 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:20.294+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:20.396 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:20.395+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:20.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:20.405 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:20.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:20.497 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:20.496+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:20.543 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:20.543 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:20.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:20.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:20.630 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:20.628+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:20.666 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:20.666 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:20.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:20.677 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T23:30:20.698 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:20.697+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:20.737 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:20.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:20.757 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:20.774 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:20.774 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:20.786 INFO:teuthology.orchestra.run.smithi139.stdout:3406 punch from 0x8f82400 to 0x8f84200, (0x1e00 bytes) 2024-09-17T23:30:20.793 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:20.796 INFO:teuthology.orchestra.run.smithi139.stdout:3408 write 0xc63de00 thru 0xc64205c (0x425d bytes) 2024-09-17T23:30:20.799 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.921290171000003, 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905665040999999, 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905627385000003, 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905609953999999, 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905575022999997, 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905543602999998, 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905526977000001, 2024-09-17T23:30:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905505282999997, 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.905493565, 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3222 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 263168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.341291222000002, 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045880157999999997, 2024-09-17T23:30:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3222 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454745+0000", 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454746+0000", 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454748+0000", 2024-09-17T23:30:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.181e-06 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454756+0000", 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454761+0000", 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7359999999999998e-06 2024-09-17T23:30:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500517+0000", 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045755245 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500564+0000", 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7647e-05 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500625+0000", 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1191000000000004e-05 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3223 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.341268755999998, 2024-09-17T23:30:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045981497000000003, 2024-09-17T23:30:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3223 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.454768+0000", 2024-09-17T23:30:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455819+0000", 2024-09-17T23:30:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010517219999999999 2024-09-17T23:30:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455824+0000", 2024-09-17T23:30:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8679999999999998e-06 2024-09-17T23:30:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455835+0000", 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0468e-05 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.455841+0000", 2024-09-17T23:30:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5799999999999997e-06 2024-09-17T23:30:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500653+0000", 2024-09-17T23:30:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044811670999999997 2024-09-17T23:30:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500696+0000", 2024-09-17T23:30:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3121999999999998e-05 2024-09-17T23:30:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.500749+0000", 2024-09-17T23:30:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3066000000000003e-05 2024-09-17T23:30:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3318 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 676864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.323930472000001, 2024-09-17T23:30:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045872493, 2024-09-17T23:30:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3318 2024-09-17T23:30:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.472106+0000", 2024-09-17T23:30:20.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473818+0000", 2024-09-17T23:30:20.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001712295 2024-09-17T23:30:20.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473823+0000", 2024-09-17T23:30:20.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3939999999999998e-06 2024-09-17T23:30:20.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473833+0000", 2024-09-17T23:30:20.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0363999999999999e-05 2024-09-17T23:30:20.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.473840+0000", 2024-09-17T23:30:20.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:30:20.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517890+0000", 2024-09-17T23:30:20.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044049873000000003 2024-09-17T23:30:20.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517931+0000", 2024-09-17T23:30:20.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1919999999999998e-05 2024-09-17T23:30:20.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.517978+0000", 2024-09-17T23:30:20.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6996999999999997e-05 2024-09-17T23:30:20.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.880 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.881 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3328 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 713728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:20.881 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:20.881 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.322022578000002, 2024-09-17T23:30:20.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045961204999999998, 2024-09-17T23:30:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:20.882 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:20.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3328 2024-09-17T23:30:20.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.883 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:20.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.474014+0000", 2024-09-17T23:30:20.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475819+0000", 2024-09-17T23:30:20.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001804719 2024-09-17T23:30:20.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475823+0000", 2024-09-17T23:30:20.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999999e-06 2024-09-17T23:30:20.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475833+0000", 2024-09-17T23:30:20.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.859e-06 2024-09-17T23:30:20.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475839+0000", 2024-09-17T23:30:20.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3829999999999997e-06 2024-09-17T23:30:20.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519770+0000", 2024-09-17T23:30:20.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.043930889000000001 2024-09-17T23:30:20.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519867+0000", 2024-09-17T23:30:20.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6744999999999996e-05 2024-09-17T23:30:20.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.519975+0000", 2024-09-17T23:30:20.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010818 2024-09-17T23:30:20.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.891 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.891 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3338 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 741376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:20.892 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:20.892 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.320037507999999, 2024-09-17T23:30:20.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045797061, 2024-09-17T23:30:20.892 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:20.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:20.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3338 2024-09-17T23:30:20.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.894 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:20.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.475999+0000", 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2100000000000001e-07 2024-09-17T23:30:20.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476000+0000", 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.094e-06 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476003+0000", 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476006+0000", 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4770000000000002e-06 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521556+0000", 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045550650999999998 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521664+0000", 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107253 2024-09-17T23:30:20.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521796+0000", 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013239299999999999 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4773.0:3339 3.5 3:a33f8b88:::rbd_data.115258a86553.0000000000000001:b [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.320027912999997, 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045949103999999998, 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4773", 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/792219417", 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3339 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.476009+0000", 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477817+0000", 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018088889999999999 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477821+0000", 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.529e-06 2024-09-17T23:30:20.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477828+0000", 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8630000000000002e-06 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.477832+0000", 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1529999999999999e-06 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521851+0000", 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044018527000000002 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521899+0000", 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8711000000000002e-05 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:41.521958+0000", 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8431999999999999e-05 2024-09-17T23:30:20.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12611 3.18 3:1c8edc2b:::rbd_data.115262cdab6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.286717193000001, 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.070059079999999996, 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:20.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12611 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509319+0000", 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.509321+0000", 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.358e-06 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553867+0000", 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.044545920000000003 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553893+0000", 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6094000000000001e-05 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553906+0000", 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3079e-05 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553972+0000", 2024-09-17T23:30:20.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6630000000000004e-05 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.554090+0000", 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117364 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.555383+0000", 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001293188 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:20.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.577545+0000", 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.022161943 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579298+0000", 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017536349999999999 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579346+0000", 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7858000000000002e-05 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.579378+0000", 2024-09-17T23:30:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2011000000000002e-05 2024-09-17T23:30:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.912 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.913 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12614 3.2 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:20.913 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:20.913 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.242096410999999, 2024-09-17T23:30:20.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.100611516, 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12614 2024-09-17T23:30:20.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553940+0000", 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.553942+0000", 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.477e-06 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599428+0000", 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.045486208 2024-09-17T23:30:20.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599454+0000", 2024-09-17T23:30:20.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6551999999999999e-05 2024-09-17T23:30:20.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599467+0000", 2024-09-17T23:30:20.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2339e-05 2024-09-17T23:30:20.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599548+0000", 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0913000000000003e-05 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.599660+0000", 2024-09-17T23:30:20.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011209699999999999 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.601086+0000", 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014260550000000001 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.612776+0000", 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.011690010000000001 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654410+0000", 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.041634517000000003 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654486+0000", 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6260999999999994e-05 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.654552+0000", 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5086999999999995e-05 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4968.0:12613 3.9 e48/19 3:92863f15:::rbd_data.115262cdab6.0000000000000003:head v 48'120, mlcod=45'117)", 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.197838316999999, 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049712148999999997, 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598198+0000", 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.598199+0000", 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5400000000000001e-07 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637181+0000", 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.038982528000000002 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637209+0000", 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7317e-05 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637219+0000", 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0372e-05 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637307+0000", 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7867000000000004e-05 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.637345+0000", 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7735999999999999e-05 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647845+0000", 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.010500012 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.647910+0000", 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5662999999999998e-05 2024-09-17T23:30:20.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12676 3.1b 3:d8a99671:::rbd_data.115262cdab6.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.191593702999999, 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.074765842999999998, 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12676 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604443+0000", 2024-09-17T23:30:20.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.604445+0000", 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1399999999999998e-06 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638570+0000", 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.034124814000000003 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638586+0000", 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6460000000000002e-05 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638611+0000", 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5216e-05 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638676+0000", 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5048999999999993e-05 2024-09-17T23:30:20.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.638792+0000", 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115233 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.640186+0000", 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013940090000000001 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657756+0000", 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017570240000000001 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679079+0000", 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.021323312000000001 2024-09-17T23:30:20.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679150+0000", 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1086000000000003e-05 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.679209+0000", 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8284e-05 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5039.0:2696 3.b 3:d04d06ca:::rbd_data.1152adcf8b1e.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 589824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:17.148959+0000", 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.6470773749999998, 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.056869014000000002, 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5039", 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1181598355", 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2696 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:20.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148959+0000", 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148959+0000", 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148960+0000", 2024-09-17T23:30:20.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6400000000000001e-07 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148964+0000", 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4839999999999998e-06 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148974+0000", 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5249999999999994e-06 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:20.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.148981+0000", 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6889999999999998e-06 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.149034+0000", 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3451e-05 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.149115+0000", 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1118000000000005e-05 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 0", 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.149213+0000", 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8055999999999998e-05 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.152526+0000", 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033133300000000002 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.205735+0000", 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.053208593999999998 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.205779+0000", 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4163999999999999e-05 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:17.205828+0000", 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8838999999999998e-05 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:20.940 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:20.941 INFO:teuthology.orchestra.run.smithi139.stdout:3410 trunc from 0xd9bf800 to 0x44f9c00 2024-09-17T23:30:20.942 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:20.797+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:20.942 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:20.943 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:20.943 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:20.944 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:20.945 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:20.945 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:20.940+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:21.043 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:21.041+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:21.052 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3.9edad2a8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.19703931799999999, 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.19706121500000001, 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:30:21.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3.9edad2a8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.19700694799999999, 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.19707110799999999, 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.057 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:30:21.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3.d8951604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.196714571, 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.19681316200000001, 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3.9edad2a8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.190890433, 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.19102239700000001, 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:21.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3.d8951604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.19084893, 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.191014146, 2024-09-17T23:30:21.069 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:30:21.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:30:21.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3.9edad2a8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.148110663, 2024-09-17T23:30:21.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.148308458, 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:30:21.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3.d8951604 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:21.077 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.148008322, 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.14823847000000001, 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:21.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 7 2024-09-17T23:30:21.081 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:21.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:21.144 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:21.142+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:21.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:30:21.189 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:21.190 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:21.190 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:21.190 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:21.190 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:21.198 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:21.245 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:21.243+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1984 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1814528~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.307 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.812119731999999, 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049003441000000002, 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1984 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492101+0000", 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1199999999999999e-07 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492103+0000", 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1540000000000001e-06 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492107+0000", 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0249999999999996e-06 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492109+0000", 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6129999999999998e-06 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540902+0000", 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048793087999999998 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.540995+0000", 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2534000000000005e-05 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541105+0000", 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000109815 2024-09-17T23:30:21.311 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:1985 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1823744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.812108438000003, 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049142263999999998, 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.312 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1985 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.492113+0000", 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493825+0000", 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017121879999999999 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493829+0000", 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2509999999999996e-06 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493836+0000", 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7000000000000002e-06 2024-09-17T23:30:21.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493840+0000", 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3220000000000003e-06 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541141+0000", 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047301395000000003 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541190+0000", 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8819999999999997e-05 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.541255+0000", 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4587999999999999e-05 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2107 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1105920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:21.316 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.810246272999997, 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048932067000000003, 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2107 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493975+0000", 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.22e-07 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493977+0000", 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493984+0000", 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0920000000000003e-06 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493989+0000", 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0370000000000002e-06 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542786+0000", 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048797034000000003 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542841+0000", 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5781e-05 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542907+0000", 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5362000000000002e-05 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.320 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2108 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1115136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.810225742999997, 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049148982000000001, 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.321 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2108 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493995+0000", 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493996+0000", 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.03e-07 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.493998+0000", 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.187e-06 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494004+0000", 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2219999999999999e-06 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494009+0000", 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6870000000000002e-06 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.542954+0000", 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048945206999999998 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543043+0000", 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9494999999999996e-05 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543144+0000", 2024-09-17T23:30:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100781 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.810205752999998, 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.326 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:21.330 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.810185873000002, 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.331 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:21.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:21.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:21.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:21.336 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.810164335000003, 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2117 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1198080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808302019999999, 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048875710000000003, 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2117 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495919+0000", 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495920+0000", 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.105e-06 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495924+0000", 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.269e-06 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495926+0000", 2024-09-17T23:30:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5830000000000001e-06 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544618+0000", 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048691940000000003 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544704+0000", 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5581000000000006e-05 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544795+0000", 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1020999999999995e-05 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2118 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1207296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808291812999997, 2024-09-17T23:30:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.04902596, 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2118 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495929+0000", 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.11e-07 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495931+0000", 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.139e-06 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495934+0000", 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1870000000000001e-06 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495936+0000", 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.542e-06 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544842+0000", 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048905954000000001 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544893+0000", 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0927999999999998e-05 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544955+0000", 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1999000000000001e-05 2024-09-17T23:30:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808281737000001, 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808271621000003, 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808259391, 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808248822000003, 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808238934000002, 2024-09-17T23:30:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808228651999997, 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.808212765, 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2129 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1308672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.806304853999997, 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048890621000000002, 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2129 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497916+0000", 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5100000000000001e-07 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497918+0000", 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4670000000000001e-06 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497922+0000", 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6799999999999999e-06 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497924+0000", 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6390000000000001e-06 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546622+0000", 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048697427000000001 2024-09-17T23:30:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546706+0000", 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4306000000000002e-05 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546807+0000", 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000100851 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.269287448, 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.267218062000001, 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.935350694, 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.416 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:21.417 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:21.415+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:21.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:21.518 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:21.517+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:21.565 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:21.566 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:21.566 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:21.566 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:21.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:21.620 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:21.618+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:21.681 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:21.681 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:21.681 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:21.689 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:21.721 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:21.719+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:21.821 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:21.820+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:21.838 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:21.838 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085847928, 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:21.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085815566000001, 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:21.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:21.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:21.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.849 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085794384000003, 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:21.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085767822000001, 2024-09-17T23:30:21.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:21.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:21.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085745649000003, 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.860 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:21.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085725736999997, 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:21.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085679061, 2024-09-17T23:30:21.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:21.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:21.873 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085652312999997, 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.874 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:21.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:21.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:21.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:21.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:21.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.880 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085624224999997, 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:21.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:21.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.085603925999997, 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.885 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:21.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:21.889 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.258805399, 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:21.890 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:21.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:21.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3143 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754855728999999, 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0144227690000001, 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3143 2024-09-17T23:30:21.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080796+0000", 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2099999999999995e-07 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080801+0000", 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6909999999999996e-06 2024-09-17T23:30:21.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080822+0000", 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1128000000000002e-05 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080830+0000", 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8280000000000003e-06 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:21.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1292999999999997e-05 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3889999999999999e-06 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083687+0000", 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027885470000000002 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083696+0000", 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3959999999999995e-06 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095020+0000", 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011324347 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095106+0000", 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5352000000000005e-05 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095218+0000", 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111877 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3144 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754801182, 2024-09-17T23:30:21.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014824425, 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3144 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:21.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080850+0000", 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080853+0000", 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2509999999999999e-06 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080861+0000", 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7840000000000004e-06 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080866+0000", 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080962+0000", 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5360999999999997e-05 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:21.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083738+0000", 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00276998 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083747+0000", 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1139999999999995e-06 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095514+0000", 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011767071 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095579+0000", 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4732000000000003e-05 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095674+0000", 2024-09-17T23:30:21.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4726000000000001e-05 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3145 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754775319, 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014984876, 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3145 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080876+0000", 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080878+0000", 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-06 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080886+0000", 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3599999999999996e-06 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8559999999999998e-06 2024-09-17T23:30:21.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081004+0000", 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112568 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081009+0000", 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083807+0000", 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002797538 2024-09-17T23:30:21.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083825+0000", 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8416000000000001e-05 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095733+0000", 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011907194 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095780+0000", 2024-09-17T23:30:21.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7942000000000003e-05 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095860+0000", 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9721999999999993e-05 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3146 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:21.917 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754752602, 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015105355, 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3146 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7e-07 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080900+0000", 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8679999999999999e-06 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080906+0000", 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7440000000000004e-06 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080911+0000", 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:21.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081059+0000", 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014868999999999999 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081067+0000", 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5650000000000004e-06 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083866+0000", 2024-09-17T23:30:21.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002799008 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083871+0000", 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9509999999999997e-06 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095889+0000", 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012018283 2024-09-17T23:30:21.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095931+0000", 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1885999999999998e-05 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096003+0000", 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2390000000000003e-05 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3147 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754734471999999, 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015262753, 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3147 2024-09-17T23:30:21.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080917+0000", 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080918+0000", 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7999999999999999e-06 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080924+0000", 2024-09-17T23:30:21.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5310000000000001e-06 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080928+0000", 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081109+0000", 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018069200000000001 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081115+0000", 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7490000000000004e-06 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083903+0000", 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027881249999999998 2024-09-17T23:30:21.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083908+0000", 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096026+0000", 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0121184919999999 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096065+0000", 2024-09-17T23:30:21.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8566999999999999e-05 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096179+0000", 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011409900000000001 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3148 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754698400000001, 2024-09-17T23:30:21.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015338839, 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3148 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080953+0000", 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4099999999999999e-07 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080955+0000", 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.21e-06 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080963+0000", 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7910000000000005e-06 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9350000000000002e-06 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081149+0000", 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018104800000000001 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081152+0000", 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083933+0000", 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027811630000000001 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083937+0000", 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0609999999999997e-06 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096195+0000", 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122576050000001 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096232+0000", 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7592999999999997e-05 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096291+0000", 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8767999999999997e-05 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3149 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.936 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754657831999999, 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154068810000001, 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3149 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2e-07 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080995+0000", 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1299999999999999e-06 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081003+0000", 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4089999999999997e-06 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081008+0000", 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8940000000000001e-06 2024-09-17T23:30:21.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081196+0000", 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018834400000000001 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081202+0000", 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083961+0000", 2024-09-17T23:30:21.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027589340000000002 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083965+0000", 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0029999999999996e-06 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096305+0000", 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012340311 2024-09-17T23:30:21.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096343+0000", 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7654000000000002e-05 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096400+0000", 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6876999999999998e-05 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.942 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3150 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.754636809000001, 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154925990000001, 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3150 2024-09-17T23:30:21.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081016+0000", 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.916e-06 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081022+0000", 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.66e-06 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081026+0000", 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3490000000000002e-06 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081221+0000", 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000194599 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081223+0000", 2024-09-17T23:30:21.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.694e-06 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083982+0000", 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002758122 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083985+0000", 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9360000000000001e-06 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096413+0000", 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124277079999999 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096450+0000", 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6449999999999998e-05 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096506+0000", 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6820999999999997e-05 2024-09-17T23:30:21.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4997.0:1730 3.1f 3:fa9e12c1:::rbd_data.1152ce7c866.0000000000000117:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.9851403540000003, 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063375800999999995, 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4997", 2024-09-17T23:30:21.949 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1359963294", 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1730 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:21.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850511+0000", 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850517+0000", 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850522+0000", 2024-09-17T23:30:21.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850526+0000", 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913749+0000", 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063222924 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:21.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913886+0000", 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013731999999999999 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:21.953 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:21.955 INFO:teuthology.orchestra.run.smithi139.stdout:3412 write 0xb547600 thru 0xb54e6af (0x70b0 bytes) 2024-09-17T23:30:21.955 INFO:teuthology.orchestra.run.smithi139.stdout:3413 read 0x6a0c000 thru 0x6a1368b (0x768c bytes) 2024-09-17T23:30:21.955 INFO:teuthology.orchestra.run.smithi139.stdout:3414 trunc from 0xb54e800 to 0xb5c600 2024-09-17T23:30:21.955 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:30:21.956 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:21.952+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:21.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:21.982 INFO:teuthology.orchestra.run.smithi139.stdout:3415 write 0x5ea7200 thru 0x5eac789 (0x558a bytes) 2024-09-17T23:30:21.983 INFO:teuthology.orchestra.run.smithi139.stdout:3416 read 0x39f2400 thru 0x39ff409 (0xd00a bytes) 2024-09-17T23:30:21.983 INFO:teuthology.orchestra.run.smithi139.stdout:3417 punch from 0x4198a00 to 0x41a4c00, (0xc200 bytes) 2024-09-17T23:30:22.002 INFO:teuthology.orchestra.run.smithi139.stdout:3419 write 0xbef2600 thru 0xbefb129 (0x8b2a bytes) 2024-09-17T23:30:22.002 INFO:teuthology.orchestra.run.smithi139.stdout:3423 punch from 0x6224000 to 0x6226e00, (0x2e00 bytes) 2024-09-17T23:30:22.003 INFO:teuthology.orchestra.run.smithi139.stdout:3424 read 0x5e2fe00 thru 0x5e387a8 (0x89a9 bytes) 2024-09-17T23:30:22.018 INFO:teuthology.orchestra.run.smithi139.stdout:3428 trunc from 0xbefb200 to 0x98ba00 2024-09-17T23:30:22.049 INFO:teuthology.orchestra.run.smithi139.stdout:3429 punch from 0x26a000 to 0x273400, (0x9400 bytes) 2024-09-17T23:30:22.053 INFO:teuthology.orchestra.run.smithi139.stdout:3432 read 0x44fc00 thru 0x45222d (0x262e bytes) 2024-09-17T23:30:22.054 INFO:teuthology.orchestra.run.smithi139.stdout:3435 read 0x918400 thru 0x91acae (0x28af bytes) 2024-09-17T23:30:22.055 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:22.053+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:22.055 INFO:teuthology.orchestra.run.smithi139.stdout:3437 read 0x68c600 thru 0x698e9e (0xc89f bytes) 2024-09-17T23:30:22.072 INFO:teuthology.orchestra.run.smithi139.stdout:3439 write 0x7871200 thru 0x788054f (0xf350 bytes) 2024-09-17T23:30:22.078 INFO:teuthology.orchestra.run.smithi139.stdout:3440 write 0x2b44e00 thru 0x2b4b66f (0x6870 bytes) 2024-09-17T23:30:22.086 INFO:teuthology.orchestra.run.smithi139.stdout:3441 trunc from 0x7880600 to 0xe895a00 2024-09-17T23:30:22.101 INFO:teuthology.orchestra.run.smithi139.stdout:3442 punch from 0x16a1a00 to 0x16aa000, (0x8600 bytes) 2024-09-17T23:30:22.108 INFO:teuthology.orchestra.run.smithi139.stdout:3443 read 0xbd61200 thru 0xbd64193 (0x2f94 bytes) 2024-09-17T23:30:22.108 INFO:teuthology.orchestra.run.smithi139.stdout:3445 read 0x8878e00 thru 0x88794a2 (0x6a3 bytes) 2024-09-17T23:30:22.108 INFO:teuthology.orchestra.run.smithi139.stdout:3446 punch from 0x9e1a400 to 0x9e1ec00, (0x4800 bytes) 2024-09-17T23:30:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:22.117 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:22.117 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:22.125 INFO:teuthology.orchestra.run.smithi139.stdout:3447 trunc from 0xe895a00 to 0xa2200 2024-09-17T23:30:22.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:22.169 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:22.168+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:22.173 INFO:teuthology.orchestra.run.smithi139.stdout:3448 write 0x6180e00 thru 0x6184c12 (0x3e13 bytes) 2024-09-17T23:30:22.199 INFO:teuthology.orchestra.run.smithi139.stdout:3452 write 0xbe97800 thru 0xbe98251 (0xa52 bytes) 2024-09-17T23:30:22.214 INFO:teuthology.orchestra.run.smithi139.stdout:3453 trunc from 0xbe98400 to 0x56b1600 2024-09-17T23:30:22.247 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:22.247 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:22.252 INFO:teuthology.orchestra.run.smithi139.stdout:3454 write 0xd275000 thru 0xd27d0b2 (0x80b3 bytes) 2024-09-17T23:30:22.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:22.257 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:22.254+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:22.257 INFO:teuthology.orchestra.run.smithi139.stdout:3456 write 0x5fb3800 thru 0x5fbdd0c (0xa50d bytes) 2024-09-17T23:30:22.258 INFO:teuthology.orchestra.run.smithi139.stdout:3458 punch from 0x8b74c00 to 0x8b79600, (0x4a00 bytes) 2024-09-17T23:30:22.258 INFO:teuthology.orchestra.run.smithi139.stdout:3465 trunc from 0xd27d200 to 0xcf65000 2024-09-17T23:30:22.277 INFO:teuthology.orchestra.run.smithi139.stdout:3466 read 0x8a88000 thru 0x8a94261 (0xc262 bytes) 2024-09-17T23:30:22.278 INFO:teuthology.orchestra.run.smithi139.stdout:3467 write 0x105f600 thru 0x1068f75 (0x9976 bytes) 2024-09-17T23:30:22.278 INFO:teuthology.orchestra.run.smithi139.stdout:3469 read 0x2096e00 thru 0x2097a8a (0xc8b bytes) 2024-09-17T23:30:22.278 INFO:teuthology.orchestra.run.smithi139.stdout:3470 clone 26 order 23 su 4194304 sc 8 2024-09-17T23:30:22.356 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:22.355+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:22.406 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.561180231, 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:30:22.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:30:22.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5516081320000001, 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:22.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:30:22.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.548077626, 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:30:22.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:30:22.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:22.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5465829200000001, 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:22.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:30:22.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5464000009999999, 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:30:22.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:30:22.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:22.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5462864999999999, 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:22.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:30:22.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:22.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5462623930000001, 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:30:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5461880620000001, 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:30:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5461645129999999, 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:30:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:30:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5429933120000001, 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:30:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:30:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:30:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5385849760000001, 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5113128689999999, 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:30:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:30:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:30:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:30:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:30:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.502130717, 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:30:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:30:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:30:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.502099224, 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:30:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:30:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.502073532, 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:30:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.50192898, 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.500360863, 2024-09-17T23:30:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:30:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.500163516, 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:30:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.500137201, 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5001088579999999, 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:30:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:22.526 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:22.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:22.529 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:22.526+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:22.629 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:22.627+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:22.677 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:22.677 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:22.677 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:22.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:22.770 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:22.768+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:22.830 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:22.828+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:22.833 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:22.833 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:22.833 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:22.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:22.931 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:22.930+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:22.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:22.992 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.517895906000007, 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:22.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:22.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:22.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.660060092999998, 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:23.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:23.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.659367799999998, 2024-09-17T23:30:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:23.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:23.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.659356218999999, 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.658828971999998, 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.658794989, 2024-09-17T23:30:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.658484396999999, 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:23.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:23.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.658458263, 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.657960356, 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.657359266, 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.656810630999999, 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.656603033, 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646987132, 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646976897999998, 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646967030999999, 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646956883000001, 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646487015000002, 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646464899000001, 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.646177204000001, 2024-09-17T23:30:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.460791034, 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:23.106 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:23.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:23.108 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:23.108 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:23.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:23.110 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:23.105+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:23.121 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:23.122 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:23.138 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:23.139 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:23.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:23.156 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:23.208 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:23.206+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:23.236 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:23.237 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:23.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:23.246 INFO:teuthology.orchestra.run.smithi139.stdout:flattening image image_client.0-clone25 2024-09-17T23:30:23.279 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:23.280 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:23.280 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T23:30:23.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T23:30:23.309 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:23.307+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:23.351 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:23.351 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:23.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:23.410 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:23.408+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:23.473 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:23.473 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:23.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:23.527 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:23.525+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:23.602 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:23.602 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:23.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:23.612 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:23.610+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:23.622 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T23:30:23.713 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:23.711+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:23.795 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:23.795 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:23.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:23.847 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:23.845+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:23.915 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:23.913+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:23.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:30:23.987 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:23.987 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:23.998 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:24.042 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:24.040+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:24.116 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:24.114+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:24.138 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.138 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:moved 2 / 82 (2.43902%) 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.74804 -> 5.8515 (expected baseline 3.62215) 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 6 -> 6 pgs (0.365854 -> 0.365854 * mean) 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 23 -> 24 pgs (1.40244 -> 1.46341 * mean) 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:30:24.177 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-17T23:30:24.178 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:30:24.178 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0004 2024-09-17T23:30:24.178 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0005 2024-09-17T23:30:24.178 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 weight 1.0000 -> 0.8917 2024-09-17T23:30:24.178 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.4953 -> 0.5434 2024-09-17T23:30:24.190 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T23:30:24.190 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T23:30:24.232 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:24.230+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:24.294 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.294 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:24.318 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:24.316+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:24.350 INFO:teuthology.orchestra.run.smithi139.stdout:3471 punch from 0x2c2b200 to 0x2c37e00, (0xcc00 bytes) 2024-09-17T23:30:24.419 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:24.417+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:24.451 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.451 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.460 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:24.520 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:24.518+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:24.607 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.608 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.616 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:24.620 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:24.618+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:24.721 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:24.719+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:24.723 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.723 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.731 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:24.822 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:24.820+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:24.838 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.838 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:24.923 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:24.922+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:24.960 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:24.960 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:24.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:25.025 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:25.023+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:25.077 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:25.077 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:25.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:25.125 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:25.123+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:25.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:25.189 INFO:teuthology.orchestra.run.smithi031.stdout:moved 6 / 82 (7.31707%) 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.8515 -> 6.11882 (expected baseline 3.62215) 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 6 -> 6 pgs (0.365854 -> 0.365854 * mean) 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 24 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0004 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0005 2024-09-17T23:30:25.190 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 weight 0.8917 -> 1.0000 2024-09-17T23:30:25.191 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 1.0000 -> 0.9081 2024-09-17T23:30:25.191 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.5434 -> 0.5879 2024-09-17T23:30:25.191 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 weight 1.0000 -> 0.8919 2024-09-17T23:30:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:30:25.204 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:25.204 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:25.205 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T23:30:25.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-17T23:30:25.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:25.258 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:25.256+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:25.298 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:25.298 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:25.316 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:25.316 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:25.328 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:25.326+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:25.349 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:25.349 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:25.382 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:25.392 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:25.392 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:25.392 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:25.392 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:25.399 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:25.399 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:25.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:25.416 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:25.416 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:25.428 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:25.426+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:25.523 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:25.523 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:25.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-17T23:30:25.526+0000 7fdcaffff640 0 --2- 172.21.15.31:0/2746063275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdcb01395a0 0x7fdcb0136030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T23:30:25.528 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:25.527+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:25.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:25.629 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:25.628+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.206753065000001, 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.348917252, 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.348224959, 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.348213377999997, 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.347686131000003, 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.347652148000002, 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.347341556000003, 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.347315422000001, 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.346817514999998, 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.346216425000001, 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.34566779, 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.345460191999997, 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335844291000001, 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335834057, 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335824189999997, 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335814042000003, 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335344173999999, 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335322058000003, 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.335034362999998, 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.149648192999997, 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:25.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:25.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:25.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:25.778 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:25.779 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:30:25.709+0000 7f44fd5a2640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-17T23:30:25.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:moved 1 / 82 (1.21951%) 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 6.11882 -> 5.67803 (expected baseline 3.62215) 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 6 -> 7 pgs (0.365854 -> 0.426829 * mean) 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 24 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:oload 1000 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0004 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0043 2024-09-17T23:30:25.781 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.5879 -> 0.6360 2024-09-17T23:30:25.782 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-17T23:30:25.782 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T23:30:25.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-17T23:30:25.783 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:25.778+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:25.880 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:25.879+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:25.936 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:25.936 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:25.936 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:25.936 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:25.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:25.981 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:25.979+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:26.083 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:26.081+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:26.093 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:26.093 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:26.093 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:26.093 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:26.093 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:26.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:26.184 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:26.182+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:26.253 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.214638975, 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.214466035000001, 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.214426164999999, 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.919921938000002, 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.672220389000003, 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.662288123000003, 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.649061074000002, 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.108531918000001, 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.108510528, 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.108493399, 2024-09-17T23:30:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.108476116999999, 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.108459088, 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.108426214000001, 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.107838508, 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.086046367000002, 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.085509178999999, 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.085023063000001, 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.012880461, 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.01285865, 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.003415779999999, 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:26.370 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:26.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:26.372 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:26.370+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:26.473 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:26.471+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3.7e6b9521 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.63706560700000003, 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.63708721199999996, 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.493 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3.7e6b9521 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.63702363299999998, 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.63708951300000005, 2024-09-17T23:30:26.497 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:30:26.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3.7e6b9521 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.63713426799999995, 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.637238003, 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:30:26.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:30:26.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 3 2024-09-17T23:30:26.506 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:26.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:26.591 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:26.589+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:26.656 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:26.656 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:26.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:26.705 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:26.703+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:26.776 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:26.774+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:26.813 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:26.813 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:26.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:26.813 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.935502290000002, 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:26.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:26.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.819 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919877159999999, 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:26.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:26.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919839504000002, 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.830 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919822072999999, 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:26.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:26.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919787141999997, 2024-09-17T23:30:26.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:26.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:26.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919755721999998, 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:26.842 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:26.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919739096000001, 2024-09-17T23:30:26.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:26.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:26.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.852 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919717402000003, 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:26.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.919705684, 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:26.858 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:26.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:26.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:26.863 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9605671649999996, 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.864 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:30:26.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:26.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.871 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9566120009999999, 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:30:26.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:30:26.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9565713430000002, 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.877 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:30:26.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:30:26.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9564146080000002, 2024-09-17T23:30:26.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:30:26.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:26.887 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9563119179999999, 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:26.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9562756959999996, 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.893 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:30:26.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:30:26.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.897 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.956243401, 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:30:26.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:30:26.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9560885900000002, 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:30:26.903 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:30:26.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:30:26.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:30:26.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:540 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1933312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859875+0000", 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9503734980000003, 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99737259099999998, 2024-09-17T23:30:26.908 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 540 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859875+0000", 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859875+0000", 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859876+0000", 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3399999999999998e-07 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859880+0000", 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3379999999999998e-06 2024-09-17T23:30:26.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859893+0000", 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2628e-05 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859901+0000", 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6310000000000003e-06 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859955+0000", 2024-09-17T23:30:26.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4647e-05 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859966+0000", 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1229999999999999e-05 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857077+0000", 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99711035299999995 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857154+0000", 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6996000000000006e-05 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857248+0000", 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3933999999999999e-05 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.913 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:547 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860085+0000", 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9501638119999996, 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99739666000000005, 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 547 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860085+0000", 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860085+0000", 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860085+0000", 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860089+0000", 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.382e-06 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860097+0000", 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8699999999999992e-06 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860103+0000", 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.178e-06 2024-09-17T23:30:26.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860155+0000", 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2073000000000001e-05 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860164+0000", 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0189999999999995e-06 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857221+0000", 2024-09-17T23:30:26.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99705694300000003 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857341+0000", 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000120436 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857481+0000", 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014015800000000001 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:561 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860143+0000", 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.9501058899999997, 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99591481999999998, 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 561 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860143+0000", 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860143+0000", 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860143+0000", 2024-09-17T23:30:26.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9299999999999998e-07 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860146+0000", 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.379e-06 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860154+0000", 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.388e-06 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860160+0000", 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6230000000000002e-06 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860209+0000", 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8977000000000002e-05 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860217+0000", 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4970000000000001e-06 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855890+0000", 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99567259200000002 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855961+0000", 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1296000000000003e-05 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856058+0000", 2024-09-17T23:30:26.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6575000000000003e-05 2024-09-17T23:30:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:26.924 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:26.930 INFO:teuthology.orchestra.run.smithi031.stdout:moved 4 / 82 (4.87805%) 2024-09-17T23:30:26.930 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:30:26.930 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 6.11882 -> 5.23832 (expected baseline 3.62215) 2024-09-17T23:30:26.930 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 6 -> 8 pgs (0.365854 -> 0.487805 * mean) 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 24 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 0.05 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0004 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0005 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.9081 -> 0.8582 2024-09-17T23:30:26.931 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.5879 -> 0.6360 2024-09-17T23:30:26.932 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 weight 0.8919 -> 0.8419 2024-09-17T23:30:26.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:26.932 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:26.923+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:26.933 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T23:30:26.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3472 write 0x740f800 thru 0x741c008 (0xc809 bytes) 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3476 read 0x34c3000 thru 0x34d06f2 (0xd6f3 bytes) 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3478 read 0xb7f5a00 thru 0xb8026c8 (0xccc9 bytes) 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3479 punch from 0x864400 to 0x865800, (0x1400 bytes) 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3480 punch from 0x4263a00 to 0x4267e00, (0x4400 bytes) 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3481 write 0x9c18200 thru 0x9c25c8f (0xda90 bytes) 2024-09-17T23:30:26.934 INFO:teuthology.orchestra.run.smithi139.stdout:3482 read 0x2d24600 thru 0x2d2862c (0x402d bytes) 2024-09-17T23:30:26.935 INFO:teuthology.orchestra.run.smithi139.stdout:3483 punch from 0xf81800 to 0xf81e00, (0x600 bytes) 2024-09-17T23:30:26.935 INFO:teuthology.orchestra.run.smithi139.stdout:3484 read 0x7826600 thru 0x78359b4 (0xf3b5 bytes) 2024-09-17T23:30:26.935 INFO:teuthology.orchestra.run.smithi139.stdout:3486 trunc from 0xcf65000 to 0x703200 2024-09-17T23:30:27.025 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:27.024+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:27.089 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:27.089 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:27.099 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:27.126 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:27.124+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:27.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:30:27.227 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:27.225+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:27.246 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:27.247 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:27.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:27.256 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:30:27.254+0000 7f44fd5a2640 -1 mgr.server reply reply (3) No such process no change 2024-09-17T23:30:27.256 INFO:teuthology.orchestra.run.smithi031.stdout:moved 2 / 82 (2.43902%) 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.23832 -> 5.60714 (expected baseline 3.62215) 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 8 -> 8 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 24 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 0.05 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0004 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0005 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.8582 -> 0.8082 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.6360 -> 0.6860 2024-09-17T23:30:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 weight 0.8419 -> 0.7919 2024-09-17T23:30:27.258 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-17T23:30:27.329 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:27.327+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:27.364 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:27.364 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.867464474999998, 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.365 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.867444595000002, 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:27.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.867423057000003, 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:27.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865540459000002, 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.379 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:27.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865530343000003, 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.383 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:27.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.387 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865518113, 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:27.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865507544000003, 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.396 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865497656000002, 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:27.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:27.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865487373999997, 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.401 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:27.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:27.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.865471487000001, 2024-09-17T23:30:27.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:27.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:27.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.32654617, 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:27.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.324476784000002, 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:27.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.992609416000001, 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:27.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:27.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:27.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.5076447379999998, 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.427 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:30:27.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:30:27.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.5076123680000002, 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:30:27.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.5073199910000001, 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:30:27.437 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:30:27.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:30:27.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:27.442 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.5014958529999998, 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.447 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.5014543500000004, 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:27.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.452 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.4587160829999997, 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.453 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:30:27.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:30:27.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.4586137419999998, 2024-09-17T23:30:27.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:27.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:27.463 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:27.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:27.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:27.466 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:27.463+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:27.566 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:27.564+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:27.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:27.604 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:27.615 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:27.615 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:27.615 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:27.620 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:27.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:27.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:27.637 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:27.655 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:27.667 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:27.665+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:27.768 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:27.766+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:27.771 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:27.771 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:27.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:27.859 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:30:27.868 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:27.867+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:27.930 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:27.930 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:27.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:27.983 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:27.982+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:28.048 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.048 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:28.070 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:28.068+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:28.171 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:28.169+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:28.204 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.204 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:28.272 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:28.270+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.362 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:28.373 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:28.371+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:28.474 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:28.472+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:28.478 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.478 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:28.574 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:28.573+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:28.594 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.594 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:28.675 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:28.674+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:28.750 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.750 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:28.802 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:28.801+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:28.872 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:28.872 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:28.877 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:28.875+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:28.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:28.978 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:28.977+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:29.036 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:29.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:29.079 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:29.078+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:29.181 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:29.180+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:29.194 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:29.194 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:29.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:29.283 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:29.281+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:29.310 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:29.310 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:29.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:29.383 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:29.381+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:29.425 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:29.425 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:29.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:29.518 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:29.516+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:29.539 INFO:teuthology.orchestra.run.smithi139.stdout:3487 write 0x8e95a00 thru 0x8e95a5c (0x5d bytes) 2024-09-17T23:30:29.545 INFO:teuthology.orchestra.run.smithi139.stdout:3488 punch from 0x4940600 to 0x494f600, (0xf000 bytes) 2024-09-17T23:30:29.546 INFO:teuthology.orchestra.run.smithi139.stdout:3490 punch from 0x6268c00 to 0x6277e00, (0xf200 bytes) 2024-09-17T23:30:29.546 INFO:teuthology.orchestra.run.smithi139.stdout:3491 punch from 0x3896e00 to 0x389da00, (0x6c00 bytes) 2024-09-17T23:30:29.549 INFO:teuthology.orchestra.run.smithi139.stdout:3492 trunc from 0x8e95c00 to 0xa9afc00 2024-09-17T23:30:29.563 INFO:teuthology.orchestra.run.smithi139.stdout:3495 trunc from 0xa9afc00 to 0xb0afe00 2024-09-17T23:30:29.579 INFO:teuthology.orchestra.run.smithi139.stdout:3502 trunc from 0xb0afe00 to 0xb486e00 2024-09-17T23:30:29.585 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:29.584+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:29.588 INFO:teuthology.orchestra.run.smithi139.stdout:3503 read 0x9f9c000 thru 0x9fab6fa (0xf6fb bytes) 2024-09-17T23:30:29.589 INFO:teuthology.orchestra.run.smithi139.stdout:3504 read 0x5e4e200 thru 0x5e56898 (0x8699 bytes) 2024-09-17T23:30:29.589 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:29.589 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:29.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:29.610 INFO:teuthology.orchestra.run.smithi139.stdout:3505 write 0xe441200 thru 0xe44612a (0x4f2b bytes) 2024-09-17T23:30:29.610 INFO:teuthology.orchestra.run.smithi139.stdout:3506 punch from 0x1705400 to 0x1711600, (0xc200 bytes) 2024-09-17T23:30:29.629 INFO:teuthology.orchestra.run.smithi139.stdout:3510 trunc from 0xe446200 to 0xf07600 2024-09-17T23:30:29.658 INFO:teuthology.orchestra.run.smithi139.stdout:3511 read 0x7e4400 thru 0x7eca25 (0x8626 bytes) 2024-09-17T23:30:29.659 INFO:teuthology.orchestra.run.smithi139.stdout:3512 read 0x997c00 thru 0x99d9f8 (0x5df9 bytes) 2024-09-17T23:30:29.659 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-17T23:30:29.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:29.688 INFO:teuthology.orchestra.run.smithi139.stdout:3514 write 0xb4d2e00 thru 0xb4d4cf4 (0x1ef5 bytes) 2024-09-17T23:30:29.688 INFO:teuthology.orchestra.run.smithi139.stdout:3517 punch from 0x8adc00 to 0x8bbc00, (0xe000 bytes) 2024-09-17T23:30:29.688 INFO:teuthology.orchestra.run.smithi139.stdout:3519 punch from 0x2eda800 to 0x2eea400, (0xfc00 bytes) 2024-09-17T23:30:29.688 INFO:teuthology.orchestra.run.smithi139.stdout:3520 read 0x691400 thru 0x699c85 (0x8886 bytes) 2024-09-17T23:30:29.690 INFO:teuthology.orchestra.run.smithi139.stdout:3521 read 0x145e00 thru 0x14d627 (0x7828 bytes) 2024-09-17T23:30:29.704 INFO:teuthology.orchestra.run.smithi139.stdout:3522 trunc from 0xb4d4e00 to 0x85fa00 2024-09-17T23:30:29.726 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:29.725+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:29.729 INFO:teuthology.orchestra.run.smithi139.stdout:3523 punch from 0x823000 to 0x826e00, (0x3e00 bytes) 2024-09-17T23:30:29.729 INFO:teuthology.orchestra.run.smithi139.stdout:3526 read 0x435000 thru 0x436227 (0x1228 bytes) 2024-09-17T23:30:29.730 INFO:teuthology.orchestra.run.smithi139.stdout:3527 read 0x63000 thru 0x6b2bd (0x82be bytes) 2024-09-17T23:30:29.730 INFO:teuthology.orchestra.run.smithi139.stdout:3528 read 0x5ef800 thru 0x5fcf87 (0xd788 bytes) 2024-09-17T23:30:29.731 INFO:teuthology.orchestra.run.smithi139.stdout:3529 read 0x7c1c00 thru 0x7c5403 (0x3804 bytes) 2024-09-17T23:30:29.731 INFO:teuthology.orchestra.run.smithi139.stdout:3533 read 0x7eb600 thru 0x7f6d0a (0xb70b bytes) 2024-09-17T23:30:29.747 INFO:teuthology.orchestra.run.smithi139.stdout:3535 trunc from 0x85fa00 to 0xdc20800 2024-09-17T23:30:29.757 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:29.757 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:29.765 INFO:teuthology.orchestra.run.smithi139.stdout:3537 punch from 0x2cae00 to 0x2d1400, (0x6600 bytes) 2024-09-17T23:30:29.765 INFO:teuthology.orchestra.run.smithi139.stdout:3538 read 0xd458c00 thru 0xd45fb00 (0x6f01 bytes) 2024-09-17T23:30:29.765 INFO:teuthology.orchestra.run.smithi139.stdout:3539 write 0x87d0e00 thru 0x87d281e (0x1a1f bytes) 2024-09-17T23:30:29.765 INFO:teuthology.orchestra.run.smithi139.stdout:3541 write 0xc84c600 thru 0xc857de9 (0xb7ea bytes) 2024-09-17T23:30:29.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:29.772 INFO:teuthology.orchestra.run.smithi139.stdout:3542 trunc from 0xdc20800 to 0x757a600 2024-09-17T23:30:29.807 INFO:teuthology.orchestra.run.smithi139.stdout:3544 trunc from 0x757a600 to 0xa019800 2024-09-17T23:30:29.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:29.809 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:29.812 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:29.810+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:29.815 INFO:teuthology.orchestra.run.smithi139.stdout:3547 write 0x3ff5200 thru 0x3ff5af3 (0x8f4 bytes) 2024-09-17T23:30:29.826 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:29.826 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:29.828 INFO:teuthology.orchestra.run.smithi139.stdout:3549 trunc from 0xa019800 to 0xba4e00 2024-09-17T23:30:29.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:29.844 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:29.858 INFO:teuthology.orchestra.run.smithi139.stdout:3551 trunc from 0xba4e00 to 0xb44d800 2024-09-17T23:30:29.862 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:29.862 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:29.873 INFO:teuthology.orchestra.run.smithi139.stdout:3554 read 0x9158000 thru 0x915ab1b (0x2b1c bytes) 2024-09-17T23:30:29.874 INFO:teuthology.orchestra.run.smithi139.stdout:3560 read 0x4f30a00 thru 0x4f34af2 (0x40f3 bytes) 2024-09-17T23:30:29.874 INFO:teuthology.orchestra.run.smithi139.stdout:3562 write 0x9f12600 thru 0x9f2030e (0xdd0f bytes) 2024-09-17T23:30:29.874 INFO:teuthology.orchestra.run.smithi139.stdout:3563 write 0x5122600 thru 0x512f176 (0xcb77 bytes) 2024-09-17T23:30:29.874 INFO:teuthology.orchestra.run.smithi139.stdout:3564 read 0xa41c00 thru 0xa4a1f0 (0x85f1 bytes) 2024-09-17T23:30:29.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:30:29.881 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:29.881 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:29.887 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:29.886+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:29.893 INFO:teuthology.orchestra.run.smithi139.stdout:3566 write 0xc0d9e00 thru 0xc0e1057 (0x7258 bytes) 2024-09-17T23:30:29.894 INFO:teuthology.orchestra.run.smithi139.stdout:3568 read 0x41f6a00 thru 0x41fd917 (0x6f18 bytes) 2024-09-17T23:30:29.894 INFO:teuthology.orchestra.run.smithi139.stdout:3570 write 0x8fc8600 thru 0x8fc9fe6 (0x19e7 bytes) 2024-09-17T23:30:29.894 INFO:teuthology.orchestra.run.smithi139.stdout:3572 punch from 0x9b39e00 to 0x9b3ba00, (0x1c00 bytes) 2024-09-17T23:30:29.894 INFO:teuthology.orchestra.run.smithi139.stdout:3575 read 0xad77a00 thru 0xad7be2c (0x442d bytes) 2024-09-17T23:30:29.894 INFO:teuthology.orchestra.run.smithi139.stdout:3581 write 0x6c05200 thru 0x6c131a0 (0xdfa1 bytes) 2024-09-17T23:30:29.896 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:29.897 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:29.897 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:29.897 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:29.897 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:29.900 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:29.900 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:29.904 INFO:teuthology.orchestra.run.smithi139.stdout:3582 trunc from 0xc0e1200 to 0x3b73400 2024-09-17T23:30:29.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:29.946 INFO:teuthology.orchestra.run.smithi139.stdout:3583 write 0x62b8000 thru 0x62bfc35 (0x7c36 bytes) 2024-09-17T23:30:29.946 INFO:teuthology.orchestra.run.smithi139.stdout:3584 punch from 0x55d5600 to 0x55d5e00, (0x800 bytes) 2024-09-17T23:30:29.946 INFO:teuthology.orchestra.run.smithi139.stdout:3588 punch from 0x3d38400 to 0x3d40400, (0x8000 bytes) 2024-09-17T23:30:29.948 INFO:teuthology.orchestra.run.smithi139.stdout:3592 trunc from 0x62bfe00 to 0x7f94c00 2024-09-17T23:30:29.965 INFO:teuthology.orchestra.run.smithi139.stdout:3593 read 0x314b600 thru 0x3152d82 (0x7783 bytes) 2024-09-17T23:30:29.965 INFO:teuthology.orchestra.run.smithi139.stdout:3594 read 0x1132a00 thru 0x113ffd4 (0xd5d5 bytes) 2024-09-17T23:30:29.973 INFO:teuthology.orchestra.run.smithi139.stdout:3595 trunc from 0x7f94c00 to 0x1a69600 2024-09-17T23:30:29.991 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:29.989+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:30.011 INFO:teuthology.orchestra.run.smithi139.stdout:3596 write 0x7071a00 thru 0x70742b0 (0x28b1 bytes) 2024-09-17T23:30:30.011 INFO:teuthology.orchestra.run.smithi139.stdout:3598 write 0x6ad7e00 thru 0x6ada707 (0x2908 bytes) 2024-09-17T23:30:30.029 INFO:teuthology.orchestra.run.smithi139.stdout:3600 write 0xbf2bc00 thru 0xbf33451 (0x7852 bytes) 2024-09-17T23:30:30.035 INFO:teuthology.orchestra.run.smithi139.stdout:3602 read 0xa37ae00 thru 0xa37d25e (0x245f bytes) 2024-09-17T23:30:30.036 INFO:teuthology.orchestra.run.smithi139.stdout:3603 read 0x5de9800 thru 0x5df37cd (0x9fce bytes) 2024-09-17T23:30:30.044 INFO:teuthology.orchestra.run.smithi139.stdout:3605 trunc from 0xbf33600 to 0x4b40400 2024-09-17T23:30:30.064 INFO:teuthology.orchestra.run.smithi139.stdout:3608 punch from 0x2746200 to 0x2747600, (0x1400 bytes) 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.580888825000002, 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:30.078 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:30.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:30.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.580868944999999, 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.083 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:30.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.580847407, 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:30.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578964808999999, 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:30.092 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.096 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578954693, 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.097 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:30.101 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578942462999997, 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:30.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578931894, 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:30.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:30.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578922005999999, 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.115 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578911724000001, 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 52.578895836999997, 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.120 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:30.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.039970519999997, 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:30.124 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:30.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.037901134000002, 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:30.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:30.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.706033766000001, 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:30.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2210690880000001, 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.141 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:30.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:30:30.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2210367180000006, 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:30:30.146 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:30:30.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.151 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2207443409999996, 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:30:30.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:30:30.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2149202030000001, 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:30:30.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:30:30.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:30:30.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.2148786999999999, 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.162 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:30.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:30:30.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1721404329999991, 2024-09-17T23:30:30.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:30:30.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1720380919999993, 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.173 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:30:30.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:30:30.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:30:30.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.178 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.178 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.178 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:30.179 INFO:teuthology.orchestra.run.smithi139.stdout:3609 write 0x8678200 thru 0x867ae5f (0x2c60 bytes) 2024-09-17T23:30:30.179 INFO:teuthology.orchestra.run.smithi139.stdout:3610 write 0x3b16400 thru 0x3b1b3ff (0x5000 bytes) 2024-09-17T23:30:30.179 INFO:teuthology.orchestra.run.smithi139.stdout:3611 read 0x440cc00 thru 0x4415f71 (0x9372 bytes) 2024-09-17T23:30:30.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:30.180 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:30.177+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:30.280 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:30.278+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:30.327 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:30.327 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:30.327 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:30.327 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:30.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:30.420 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:30.418+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:30.481 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:30.480+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:30.483 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:30.483 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:30.483 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:30.483 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:30.483 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:30.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:30.583 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:30.581+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.888087089000003, 2024-09-17T23:30:30.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:30.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.645 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.888054726999997, 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:30.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:30.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:30.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.888033544999999, 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.888006982999997, 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:30.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:30.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:30.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.887984809999999, 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.661 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:30.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.665 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.887964898, 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:30.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.670 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.887918222000003, 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:30.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:30.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.887891474, 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:30.675 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:30.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.887863385999999, 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:30.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.887843087, 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:30.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:30.689 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.061044559999999, 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:30.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:30.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3143 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.695 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.557094889999998, 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0144227690000001, 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3143 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:30.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080796+0000", 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2099999999999995e-07 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080801+0000", 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6909999999999996e-06 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080822+0000", 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1128000000000002e-05 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080830+0000", 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8280000000000003e-06 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1292999999999997e-05 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3889999999999999e-06 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083687+0000", 2024-09-17T23:30:30.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027885470000000002 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083696+0000", 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3959999999999995e-06 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095020+0000", 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011324347 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095106+0000", 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5352000000000005e-05 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095218+0000", 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111877 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3144 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.557040343000001, 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014824425, 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3144 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:30.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080850+0000", 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080853+0000", 2024-09-17T23:30:30.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2509999999999999e-06 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080861+0000", 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7840000000000004e-06 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080866+0000", 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080962+0000", 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5360999999999997e-05 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083738+0000", 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00276998 2024-09-17T23:30:30.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083747+0000", 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1139999999999995e-06 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095514+0000", 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011767071 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095579+0000", 2024-09-17T23:30:30.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4732000000000003e-05 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095674+0000", 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4726000000000001e-05 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3145 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:30.707 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.557014479999999, 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014984876, 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3145 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:30.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080876+0000", 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080878+0000", 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-06 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080886+0000", 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3599999999999996e-06 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8559999999999998e-06 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081004+0000", 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112568 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081009+0000", 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083807+0000", 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002797538 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083825+0000", 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8416000000000001e-05 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095733+0000", 2024-09-17T23:30:30.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011907194 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095780+0000", 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7942000000000003e-05 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095860+0000", 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9721999999999993e-05 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3146 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.556991762999999, 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015105355, 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3146 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7e-07 2024-09-17T23:30:30.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080900+0000", 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8679999999999999e-06 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080906+0000", 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7440000000000004e-06 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080911+0000", 2024-09-17T23:30:30.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081059+0000", 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014868999999999999 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081067+0000", 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5650000000000004e-06 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083866+0000", 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002799008 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083871+0000", 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9509999999999997e-06 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095889+0000", 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012018283 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095931+0000", 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1885999999999998e-05 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096003+0000", 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2390000000000003e-05 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.719 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3147 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.556973632999998, 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015262753, 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3147 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080917+0000", 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080918+0000", 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7999999999999999e-06 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080924+0000", 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5310000000000001e-06 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080928+0000", 2024-09-17T23:30:30.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081109+0000", 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018069200000000001 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081115+0000", 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7490000000000004e-06 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083903+0000", 2024-09-17T23:30:30.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027881249999999998 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083908+0000", 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096026+0000", 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0121184919999999 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096065+0000", 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8566999999999999e-05 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096179+0000", 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011409900000000001 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3148 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.725 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.556937561000002, 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015338839, 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3148 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080953+0000", 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4099999999999999e-07 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080955+0000", 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.21e-06 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080963+0000", 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7910000000000005e-06 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:30.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9350000000000002e-06 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081149+0000", 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018104800000000001 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081152+0000", 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083933+0000", 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027811630000000001 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083937+0000", 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0609999999999997e-06 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096195+0000", 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122576050000001 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096232+0000", 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7592999999999997e-05 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096291+0000", 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8767999999999997e-05 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.731 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3149 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.556896992999999, 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154068810000001, 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3149 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2e-07 2024-09-17T23:30:30.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080995+0000", 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1299999999999999e-06 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081003+0000", 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4089999999999997e-06 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081008+0000", 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8940000000000001e-06 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081196+0000", 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018834400000000001 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081202+0000", 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083961+0000", 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027589340000000002 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083965+0000", 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0029999999999996e-06 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096305+0000", 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012340311 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096343+0000", 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7654000000000002e-05 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096400+0000", 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6876999999999998e-05 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3150 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.55687597, 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154925990000001, 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3150 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:30.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081016+0000", 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.916e-06 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081022+0000", 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.66e-06 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081026+0000", 2024-09-17T23:30:30.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3490000000000002e-06 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081221+0000", 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000194599 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081223+0000", 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.694e-06 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083982+0000", 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002758122 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083985+0000", 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9360000000000001e-06 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096413+0000", 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124277079999999 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:30.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096450+0000", 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6449999999999998e-05 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096506+0000", 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6820999999999997e-05 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4997.0:1730 3.1f 3:fa9e12c1:::rbd_data.1152ce7c866.0000000000000117:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:30.743 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.787379515000001, 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063375800999999995, 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4997", 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1359963294", 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1730 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850511+0000", 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850517+0000", 2024-09-17T23:30:30.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850522+0000", 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850526+0000", 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913749+0000", 2024-09-17T23:30:30.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063222924 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913886+0000", 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013731999999999999 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:30.747 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:30.749 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:30.746+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:30.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:30.849 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:30.847+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:30.868 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:30:30.873 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:30.873 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:30.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5367.0:1253 3.a 3.4ea4af4a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:30.089248+0000", 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.78290247400000001, 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.782921439, 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5367", 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1253 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089248+0000", 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089248+0000", 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089249+0000", 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3400000000000001e-06 2024-09-17T23:30:30.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089277+0000", 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7772000000000001e-05 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089288+0000", 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1487e-05 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:30.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089296+0000", 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2930000000000003e-06 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089360+0000", 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4750000000000002e-05 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:30.089373+0000", 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2497000000000001e-05 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:30.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:30.878 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:30.878 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-17T23:30:30.878 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:30.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:30.890 INFO:teuthology.orchestra.run.smithi139.stdout:3612 write 0xe53dc00 thru 0xe546aac (0x8ead bytes) 2024-09-17T23:30:30.891 INFO:teuthology.orchestra.run.smithi139.stdout:3614 punch from 0x3e9e000 to 0x3eaa600, (0xc600 bytes) 2024-09-17T23:30:30.891 INFO:teuthology.orchestra.run.smithi139.stdout:3617 punch from 0x5b89c00 to 0x5b96e00, (0xd200 bytes) 2024-09-17T23:30:30.891 INFO:teuthology.orchestra.run.smithi139.stdout:3619 punch from 0x8f62e00 to 0x8f6f000, (0xc200 bytes) 2024-09-17T23:30:30.891 INFO:teuthology.orchestra.run.smithi139.stdout:3620 write 0xd352e00 thru 0xd3587e0 (0x59e1 bytes) 2024-09-17T23:30:30.895 INFO:teuthology.orchestra.run.smithi139.stdout:3626 trunc from 0xe546c00 to 0xca1fa00 2024-09-17T23:30:30.925 INFO:teuthology.orchestra.run.smithi139.stdout:3629 read 0x54fca00 thru 0x5503b56 (0x7157 bytes) 2024-09-17T23:30:30.928 INFO:teuthology.orchestra.run.smithi139.stdout:3631 trunc from 0xca1fa00 to 0xa266c00 2024-09-17T23:30:30.950 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:30.948+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:30.950 INFO:teuthology.orchestra.run.smithi139.stdout:3632 punch from 0x816dc00 to 0x816ea00, (0xe00 bytes) 2024-09-17T23:30:30.950 INFO:teuthology.orchestra.run.smithi139.stdout:3633 read 0x5ff8600 thru 0x6002dc1 (0xa7c2 bytes) 2024-09-17T23:30:30.955 INFO:teuthology.orchestra.run.smithi139.stdout:3639 trunc from 0xa266c00 to 0x67a2400 2024-09-17T23:30:30.974 INFO:teuthology.orchestra.run.smithi139.stdout:3640 punch from 0x63fda00 to 0x640c400, (0xea00 bytes) 2024-09-17T23:30:30.975 INFO:teuthology.orchestra.run.smithi139.stdout:3643 punch from 0x1771800 to 0x1776200, (0x4a00 bytes) 2024-09-17T23:30:30.984 INFO:teuthology.orchestra.run.smithi139.stdout:3645 trunc from 0x67a2400 to 0xd793a00 2024-09-17T23:30:30.993 INFO:teuthology.orchestra.run.smithi139.stdout:3648 write 0x9b66000 thru 0x9b70cbe (0xacbf bytes) 2024-09-17T23:30:30.995 INFO:teuthology.orchestra.run.smithi139.stdout:3653 punch from 0x18d8400 to 0x18df200, (0x6e00 bytes) 2024-09-17T23:30:30.995 INFO:teuthology.orchestra.run.smithi139.stdout:3655 write 0x5301c00 thru 0x53041ef (0x25f0 bytes) 2024-09-17T23:30:30.995 INFO:teuthology.orchestra.run.smithi139.stdout:3658 write 0x7e89600 thru 0x7e98e28 (0xf829 bytes) 2024-09-17T23:30:30.995 INFO:teuthology.orchestra.run.smithi139.stdout:3661 read 0x56e9400 thru 0x56ee1b8 (0x4db9 bytes) 2024-09-17T23:30:30.995 INFO:teuthology.orchestra.run.smithi139.stdout:3663 write 0x95ed000 thru 0x95f74b8 (0xa4b9 bytes) 2024-09-17T23:30:30.997 INFO:teuthology.orchestra.run.smithi139.stdout:3664 trunc from 0xd793a00 to 0xbdca600 2024-09-17T23:30:31.000 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:31.000 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:31.001 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:31.001 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:31.001 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:31.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:31.015 INFO:teuthology.orchestra.run.smithi139.stdout:3667 write 0x4b73400 thru 0x4b8051f (0xd120 bytes) 2024-09-17T23:30:31.024 INFO:teuthology.orchestra.run.smithi139.stdout:3668 trunc from 0xbdca600 to 0x476c800 2024-09-17T23:30:31.050 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:31.049+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:31.063 INFO:teuthology.orchestra.run.smithi139.stdout:3671 trunc from 0x476c800 to 0x9ac0000 2024-09-17T23:30:31.072 INFO:teuthology.orchestra.run.smithi139.stdout:3672 read 0x2016a00 thru 0x2017a71 (0x1072 bytes) 2024-09-17T23:30:31.078 INFO:teuthology.orchestra.run.smithi139.stdout:3674 read 0x4c30200 thru 0x4c3de41 (0xdc42 bytes) 2024-09-17T23:30:31.082 INFO:teuthology.orchestra.run.smithi139.stdout:3675 trunc from 0x9ac0000 to 0x661ac00 2024-09-17T23:30:31.093 INFO:teuthology.orchestra.run.smithi139.stdout:3676 punch from 0x4e3f000 to 0x4e45400, (0x6400 bytes) 2024-09-17T23:30:31.094 INFO:teuthology.orchestra.run.smithi139.stdout:3679 punch from 0x4b0ec00 to 0x4b15c00, (0x7000 bytes) 2024-09-17T23:30:31.112 INFO:teuthology.orchestra.run.smithi139.stdout:3680 write 0x6b4e000 thru 0x6b5b6e2 (0xd6e3 bytes) 2024-09-17T23:30:31.112 INFO:teuthology.orchestra.run.smithi139.stdout:3682 read 0x682ca00 thru 0x682cd02 (0x303 bytes) 2024-09-17T23:30:31.118 INFO:teuthology.orchestra.run.smithi139.stdout:3683 trunc from 0x6b5b800 to 0xb5b3000 2024-09-17T23:30:31.146 INFO:teuthology.orchestra.run.smithi139.stdout:3684 write 0xc768000 thru 0xc7691ff (0x1200 bytes) 2024-09-17T23:30:31.147 INFO:teuthology.orchestra.run.smithi139.stdout:3686 trunc from 0xc769200 to 0xc762c00 2024-09-17T23:30:31.155 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:31.153+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:31.159 INFO:teuthology.orchestra.run.smithi139.stdout:3687 read 0x4ee3c00 thru 0x4ef03ea (0xc7eb bytes) 2024-09-17T23:30:31.165 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.317746025, 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:30:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:30:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.308173926, 2024-09-17T23:30:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:30:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:30:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.30464342, 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:30:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.303148714000001, 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:30:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.302965795, 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:30:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.302852294000001, 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:30:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.302828186999999, 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.302753856000001, 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:30:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:30:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.302730306999999, 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:30:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.299559106, 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.295150769999999, 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:30:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:30:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.267878662999999, 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:30:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.258696511, 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:30:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:30:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:30:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.258665018, 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:30:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.258639326000001, 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.258494774000001, 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:30:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:30:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256926656999999, 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:30:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:30:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:30:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256729310000001, 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:30:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256702994999999, 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:30:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.256674651999999, 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:30:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:30:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:30:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.278 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:31.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3688 read 0x6049c00 thru 0x604b6f3 (0x1af4 bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3690 punch from 0x51cac00 to 0x51d2e00, (0x8200 bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3691 read 0x3715e00 thru 0x371d2f6 (0x74f7 bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3693 write 0xca4f800 thru 0xca5052c (0xd2d bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3694 read 0x15bac00 thru 0x15c2a98 (0x7e99 bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3696 read 0x2532400 thru 0x253e4fb (0xc0fc bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3698 read 0x87ac800 thru 0x87b17f2 (0x4ff3 bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3699 write 0x33a7a00 thru 0x33b734d (0xf94e bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3700 read 0x3f93800 thru 0x3f959ac (0x21ad bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3702 trunc from 0xca50600 to 0x4134800 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3704 write 0x6f43600 thru 0x6f4cb2b (0x952c bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3711 write 0x3479800 thru 0x34874a7 (0xdca8 bytes) 2024-09-17T23:30:31.281 INFO:teuthology.orchestra.run.smithi139.stdout:3714 read 0x472c600 thru 0x472d222 (0xc23 bytes) 2024-09-17T23:30:31.282 INFO:teuthology.orchestra.run.smithi139.stdout:3715 write 0xe71ba00 thru 0xe72b54c (0xfb4d bytes) 2024-09-17T23:30:31.282 INFO:teuthology.orchestra.run.smithi139.stdout:3716 write 0xc891800 thru 0xc89bb12 (0xa313 bytes) 2024-09-17T23:30:31.282 INFO:teuthology.orchestra.run.smithi139.stdout:3717 write 0x2624c00 thru 0x262bed4 (0x72d5 bytes) 2024-09-17T23:30:31.282 INFO:teuthology.orchestra.run.smithi139.stdout:3723 write 0x2fb0400 thru 0x2fb4931 (0x4532 bytes) 2024-09-17T23:30:31.282 INFO:teuthology.orchestra.run.smithi139.stdout:3724 trunc from 0xe72b600 to 0xc0dc200 2024-09-17T23:30:31.282 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:31.279+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:31.296 INFO:teuthology.orchestra.run.smithi139.stdout:3725 read 0x26dac00 thru 0x26e5c6f (0xb070 bytes) 2024-09-17T23:30:31.297 INFO:teuthology.orchestra.run.smithi139.stdout:3727 punch from 0x5f57c00 to 0x5f64200, (0xc600 bytes) 2024-09-17T23:30:31.297 INFO:teuthology.orchestra.run.smithi139.stdout:3730 punch from 0xb930a00 to 0xb932e00, (0x2400 bytes) 2024-09-17T23:30:31.311 INFO:teuthology.orchestra.run.smithi139.stdout:3731 write 0xec0b800 thru 0xec1964e (0xde4f bytes) 2024-09-17T23:30:31.312 INFO:teuthology.orchestra.run.smithi139.stdout:3732 read 0x609b000 thru 0x60a4179 (0x917a bytes) 2024-09-17T23:30:31.313 INFO:teuthology.orchestra.run.smithi139.stdout:3733 write 0xa8e5a00 thru 0xa8f0a84 (0xb085 bytes) 2024-09-17T23:30:31.313 INFO:teuthology.orchestra.run.smithi139.stdout:3735 write 0xc0dd800 thru 0xc0e2e50 (0x5651 bytes) 2024-09-17T23:30:31.313 INFO:teuthology.orchestra.run.smithi139.stdout:3737 write 0x43b7400 thru 0x43bf1f6 (0x7df7 bytes) 2024-09-17T23:30:31.314 INFO:teuthology.orchestra.run.smithi139.stdout:3738 punch from 0x48e0800 to 0x48e0c00, (0x400 bytes) 2024-09-17T23:30:31.314 INFO:teuthology.orchestra.run.smithi139.stdout:3740 read 0x6965400 thru 0x6967936 (0x2537 bytes) 2024-09-17T23:30:31.317 INFO:teuthology.orchestra.run.smithi139.stdout:3741 trunc from 0xec19800 to 0xc578200 2024-09-17T23:30:31.339 INFO:teuthology.orchestra.run.smithi139.stdout:3743 trunc from 0xc578200 to 0xd2b9a00 2024-09-17T23:30:31.355 INFO:teuthology.orchestra.run.smithi139.stdout:3744 read 0xb627200 thru 0xb633e2b (0xcc2c bytes) 2024-09-17T23:30:31.356 INFO:teuthology.orchestra.run.smithi139.stdout:3746 read 0xced9200 thru 0xcee7803 (0xe604 bytes) 2024-09-17T23:30:31.356 INFO:teuthology.orchestra.run.smithi139.stdout:3747 write 0x26db800 thru 0x26e46a7 (0x8ea8 bytes) 2024-09-17T23:30:31.357 INFO:teuthology.orchestra.run.smithi139.stdout:3749 read 0xbbb6a00 thru 0xbbbccf0 (0x62f1 bytes) 2024-09-17T23:30:31.358 INFO:teuthology.orchestra.run.smithi139.stdout:3750 trunc from 0xd2b9a00 to 0xdcb8400 2024-09-17T23:30:31.368 INFO:teuthology.orchestra.run.smithi139.stdout:3751 write 0x4c71600 thru 0x4c7c5cd (0xafce bytes) 2024-09-17T23:30:31.369 INFO:teuthology.orchestra.run.smithi139.stdout:3756 write 0x49f4800 thru 0x49f4a8e (0x28f bytes) 2024-09-17T23:30:31.370 INFO:teuthology.orchestra.run.smithi139.stdout:3758 write 0xae43200 thru 0xae43d55 (0xb56 bytes) 2024-09-17T23:30:31.370 INFO:teuthology.orchestra.run.smithi139.stdout:3759 write 0xb6c4000 thru 0xb6cd42c (0x942d bytes) 2024-09-17T23:30:31.370 INFO:teuthology.orchestra.run.smithi139.stdout:3760 read 0xdb51600 thru 0xdb571a3 (0x5ba4 bytes) 2024-09-17T23:30:31.370 INFO:teuthology.orchestra.run.smithi139.stdout:3763 punch from 0xa5800 to 0xb2800, (0xd000 bytes) 2024-09-17T23:30:31.371 INFO:teuthology.orchestra.run.smithi139.stdout:3766 punch from 0xdc83c00 to 0xdc90600, (0xca00 bytes) 2024-09-17T23:30:31.371 INFO:teuthology.orchestra.run.smithi139.stdout:3767 write 0x3030400 thru 0x303dc5b (0xd85c bytes) 2024-09-17T23:30:31.371 INFO:teuthology.orchestra.run.smithi139.stdout:3771 read 0xbd3ea00 thru 0xbd488ad (0x9eae bytes) 2024-09-17T23:30:31.379 INFO:teuthology.orchestra.run.smithi139.stdout:3772 trunc from 0xdcb8400 to 0x75a1800 2024-09-17T23:30:31.381 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:31.380+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:31.396 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:31.396 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:31.398 INFO:teuthology.orchestra.run.smithi139.stdout:3773 read 0x7017200 thru 0x701b3d2 (0x41d3 bytes) 2024-09-17T23:30:31.399 INFO:teuthology.orchestra.run.smithi139.stdout:3774 read 0x70bbe00 thru 0x70caa7f (0xec80 bytes) 2024-09-17T23:30:31.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:31.407 INFO:teuthology.orchestra.run.smithi139.stdout:3777 trunc from 0x75a1800 to 0xe3fda00 2024-09-17T23:30:31.416 INFO:teuthology.orchestra.run.smithi139.stdout:3778 write 0x1629a00 thru 0x162e4a7 (0x4aa8 bytes) 2024-09-17T23:30:31.429 INFO:teuthology.orchestra.run.smithi139.stdout:3782 trunc from 0xe3fda00 to 0x7cb7400 2024-09-17T23:30:31.442 INFO:teuthology.orchestra.run.smithi139.stdout:3785 read 0x269c800 thru 0x26a2091 (0x5892 bytes) 2024-09-17T23:30:31.442 INFO:teuthology.orchestra.run.smithi139.stdout:3789 write 0x3a13800 thru 0x3a22109 (0xe90a bytes) 2024-09-17T23:30:31.443 INFO:teuthology.orchestra.run.smithi139.stdout:3792 punch from 0x57bd200 to 0x57c7a00, (0xa800 bytes) 2024-09-17T23:30:31.443 INFO:teuthology.orchestra.run.smithi139.stdout:3793 punch from 0x10c3e00 to 0x10c7800, (0x3a00 bytes) 2024-09-17T23:30:31.448 INFO:teuthology.orchestra.run.smithi139.stdout:3794 trunc from 0x7cb7400 to 0x3e33e00 2024-09-17T23:30:31.483 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:31.481+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:31.517 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:31.517 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:31.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:31.583 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:31.582+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.201245096999997, 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.343409284000003, 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.342716991000003, 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.342705410000001, 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.342178163, 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.342144179999998, 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.341833588, 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.341807453999998, 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.341309547000002, 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.340708456999998, 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.340159821999997, 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.339952224000001, 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.330336322999997, 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.330326089000003, 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.330316222, 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.330306073999999, 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.329836206000003, 2024-09-17T23:30:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.329814089999999, 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.329526395000002, 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.144140225000001, 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:31.771 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:31.769+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:31.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3796 write 0xdc83a00 thru 0xdc91c2c (0xe22d bytes) 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3798 read 0xa2e3200 thru 0xa2e707a (0x3e7b bytes) 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3800 punch from 0xd30f600 to 0xd31a800, (0xb200 bytes) 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3802 read 0xc510800 thru 0xc5172b1 (0x6ab2 bytes) 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3804 read 0x15d9800 thru 0x15e50cf (0xb8d0 bytes) 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3805 punch from 0x1188200 to 0x118ba00, (0x3800 bytes) 2024-09-17T23:30:31.774 INFO:teuthology.orchestra.run.smithi139.stdout:3806 write 0x7cbe600 thru 0x7cc767d (0x907e bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3807 trunc from 0xdc91e00 to 0xbb77400 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3808 write 0x7ad1200 thru 0x7adabf6 (0x99f7 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3810 read 0x2053c00 thru 0x205fd22 (0xc123 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3814 read 0x9a9b800 thru 0x9aa162c (0x5e2d bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3816 read 0x3848c00 thru 0x384b9d4 (0x2dd5 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3817 read 0x53e5400 thru 0x53e6b23 (0x1724 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3818 write 0xc46e200 thru 0xc4770e1 (0x8ee2 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3819 trunc from 0xc477200 to 0x7f3c600 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3820 read 0x18d6400 thru 0x18e5804 (0xf405 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3823 write 0xc47a00 thru 0xc52148 (0xa749 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3826 write 0x11afc00 thru 0x11b0027 (0x428 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3830 punch from 0x4c2dc00 to 0x4c3a400, (0xc800 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3831 read 0x23eda00 thru 0x23f9968 (0xbf69 bytes) 2024-09-17T23:30:31.775 INFO:teuthology.orchestra.run.smithi139.stdout:3833 trunc from 0x7f3c600 to 0x75ba00 2024-09-17T23:30:31.803 INFO:teuthology.orchestra.run.smithi139.stdout:3834 trunc from 0x75ba00 to 0x914f800 2024-09-17T23:30:31.813 INFO:teuthology.orchestra.run.smithi139.stdout:3838 read 0x1fa4200 thru 0x1fa7ab2 (0x38b3 bytes) 2024-09-17T23:30:31.813 INFO:teuthology.orchestra.run.smithi139.stdout:3839 write 0x4462a00 thru 0x446748a (0x4a8b bytes) 2024-09-17T23:30:31.814 INFO:teuthology.orchestra.run.smithi139.stdout:3840 trunc from 0x914f800 to 0x9a4e600 2024-09-17T23:30:31.826 INFO:teuthology.orchestra.run.smithi139.stdout:3841 read 0x2ae6800 thru 0x2aef83f (0x9040 bytes) 2024-09-17T23:30:31.827 INFO:teuthology.orchestra.run.smithi139.stdout:3844 write 0x5d21c00 thru 0x5d2fe26 (0xe227 bytes) 2024-09-17T23:30:31.827 INFO:teuthology.orchestra.run.smithi139.stdout:3845 write 0x82b9000 thru 0x82bc8c4 (0x38c5 bytes) 2024-09-17T23:30:31.827 INFO:teuthology.orchestra.run.smithi139.stdout:3846 write 0x8bede00 thru 0x8bf76d2 (0x98d3 bytes) 2024-09-17T23:30:31.827 INFO:teuthology.orchestra.run.smithi139.stdout:3847 read 0x52d200 thru 0x52e4d8 (0x12d9 bytes) 2024-09-17T23:30:31.828 INFO:teuthology.orchestra.run.smithi139.stdout:3848 read 0x7a3c400 thru 0x7a41600 (0x5201 bytes) 2024-09-17T23:30:31.828 INFO:teuthology.orchestra.run.smithi139.stdout:3850 punch from 0x1933a00 to 0x1934800, (0xe00 bytes) 2024-09-17T23:30:31.828 INFO:teuthology.orchestra.run.smithi139.stdout:3853 read 0x336d000 thru 0x336e153 (0x1154 bytes) 2024-09-17T23:30:31.828 INFO:teuthology.orchestra.run.smithi139.stdout:3855 read 0x3223a00 thru 0x322581b (0x1e1c bytes) 2024-09-17T23:30:31.842 INFO:teuthology.orchestra.run.smithi139.stdout:3856 write 0xae09200 thru 0xae1001c (0x6e1d bytes) 2024-09-17T23:30:31.843 INFO:teuthology.orchestra.run.smithi139.stdout:3858 read 0x3ce2800 thru 0x3cea14a (0x794b bytes) 2024-09-17T23:30:31.843 INFO:teuthology.orchestra.run.smithi139.stdout:3859 read 0x261fe00 thru 0x2623816 (0x3a17 bytes) 2024-09-17T23:30:31.843 INFO:teuthology.orchestra.run.smithi139.stdout:3862 read 0x76a1400 thru 0x76aaebe (0x9abf bytes) 2024-09-17T23:30:31.843 INFO:teuthology.orchestra.run.smithi139.stdout:3863 write 0x6428600 thru 0x643538a (0xcd8b bytes) 2024-09-17T23:30:31.844 INFO:teuthology.orchestra.run.smithi139.stdout:3865 punch from 0x36cf600 to 0x36d6600, (0x7000 bytes) 2024-09-17T23:30:31.844 INFO:teuthology.orchestra.run.smithi139.stdout:3869 read 0x3fc2e00 thru 0x3fc66f6 (0x38f7 bytes) 2024-09-17T23:30:31.844 INFO:teuthology.orchestra.run.smithi139.stdout:3870 read 0x1027600 thru 0x102d203 (0x5c04 bytes) 2024-09-17T23:30:31.845 INFO:teuthology.orchestra.run.smithi139.stdout:3872 read 0x61a1a00 thru 0x61a9e27 (0x8428 bytes) 2024-09-17T23:30:31.845 INFO:teuthology.orchestra.run.smithi139.stdout:3876 read 0x311be00 thru 0x311f8ae (0x3aaf bytes) 2024-09-17T23:30:31.846 INFO:teuthology.orchestra.run.smithi139.stdout:3877 trunc from 0xae10200 to 0xc5c0400 2024-09-17T23:30:31.858 INFO:teuthology.orchestra.run.smithi139.stdout:3880 read 0xb96b000 thru 0xb978b57 (0xdb58 bytes) 2024-09-17T23:30:31.859 INFO:teuthology.orchestra.run.smithi139.stdout:3883 write 0x84c8600 thru 0x84cd653 (0x5054 bytes) 2024-09-17T23:30:31.860 INFO:teuthology.orchestra.run.smithi139.stdout:3887 punch from 0x8e07a00 to 0x8e0c000, (0x4600 bytes) 2024-09-17T23:30:31.860 INFO:teuthology.orchestra.run.smithi139.stdout:3888 punch from 0x1ce5200 to 0x1ce5c00, (0xa00 bytes) 2024-09-17T23:30:31.860 INFO:teuthology.orchestra.run.smithi139.stdout:3889 write 0xa69000 thru 0xa783ef (0xf3f0 bytes) 2024-09-17T23:30:31.860 INFO:teuthology.orchestra.run.smithi139.stdout:3890 write 0x3c6c400 thru 0x3c71b6f (0x5770 bytes) 2024-09-17T23:30:31.860 INFO:teuthology.orchestra.run.smithi139.stdout:3892 write 0xa3bca00 thru 0xa3ca5d5 (0xdbd6 bytes) 2024-09-17T23:30:31.863 INFO:teuthology.orchestra.run.smithi139.stdout:3893 read 0x421aa00 thru 0x422a246 (0xf847 bytes) 2024-09-17T23:30:31.872 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:31.870+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:31.877 INFO:teuthology.orchestra.run.smithi139.stdout:3895 trunc from 0xc5c0400 to 0x9fac00 2024-09-17T23:30:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:31.890 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:31.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:31.904 INFO:teuthology.orchestra.run.smithi139.stdout:3897 read 0x4fc600 thru 0x509398 (0xcd99 bytes) 2024-09-17T23:30:31.906 INFO:teuthology.orchestra.run.smithi139.stdout:3898 punch from 0xf9000 to 0xf9c00, (0xc00 bytes) 2024-09-17T23:30:31.906 INFO:teuthology.orchestra.run.smithi139.stdout:3899 read 0x39a800 thru 0x39b8ef (0x10f0 bytes) 2024-09-17T23:30:31.908 INFO:teuthology.orchestra.run.smithi139.stdout:3900 punch from 0x312800 to 0x318c00, (0x6400 bytes) 2024-09-17T23:30:31.908 INFO:teuthology.orchestra.run.smithi139.stdout:3902 punch from 0x2f8400 to 0x305000, (0xcc00 bytes) 2024-09-17T23:30:31.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:31.926 INFO:teuthology.orchestra.run.smithi139.stdout:3905 write 0x7ddf600 thru 0x7de693e (0x733f bytes) 2024-09-17T23:30:31.927 INFO:teuthology.orchestra.run.smithi139.stdout:3906 read 0x780a00 thru 0x78f5b2 (0xebb3 bytes) 2024-09-17T23:30:31.928 INFO:teuthology.orchestra.run.smithi139.stdout:3907 clone 27 order 22 su 1048576 sc 7 2024-09-17T23:30:31.973 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:31.971+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:32.009 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:32.009 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:32.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:32.068 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:32.068 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:32.074 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:32.072+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:32.086 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:32.086 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:32.118 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:32.118 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:32.142 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:32.143 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:32.159 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:32.159 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:32.176 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:32.176 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:32.200 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:32.198+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:32.263 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.263 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.270 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:32.270 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:32.270 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T23:30:32.270 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T23:30:32.270 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T23:30:32.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-17T23:30:32.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:32.314 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:32.312+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:32.378 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:32.376+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:32.419 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:32.419 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T23:30:32.419 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:32.420 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.420 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:32.479 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:32.477+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:32.536 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.536 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:32.580 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:32.579+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:32.681 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:32.679+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:32.692 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.692 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:32.782 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:32.780+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:32.807 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.807 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:32.883 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:32.881+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:32.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:30:32.926 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:32.927 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:32.935 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:32.956 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone26 intact 2024-09-17T23:30:32.984 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:32.982+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:33.085 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:33.083+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:33.086 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.086 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:33.185 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:33.183+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:33.204 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.205 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:33.286 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:33.284+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:33.361 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.361 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:33.413 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:33.412+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:33.482 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.483 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.487 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:33.485+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:33.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:33.498 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T23:30:33.587 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:33.586+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:33.642 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.642 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:33.693 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:33.691+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:33.789 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:33.788+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:33.805 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.805 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:33.880 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:30:33.890 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:33.889+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:33.965 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:33.966 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:33.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:33.991 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:33.989+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:34.035 INFO:teuthology.orchestra.run.smithi139.stdout:3913 punch from 0xd17e00 to 0xd1f400, (0x7600 bytes) 2024-09-17T23:30:34.054 INFO:teuthology.orchestra.run.smithi139.stdout:3914 trunc from 0x7de6a00 to 0x8a5e00 2024-09-17T23:30:34.078 INFO:teuthology.orchestra.run.smithi139.stdout:3915 read 0x4a4200 thru 0x4aedbf (0xabc0 bytes) 2024-09-17T23:30:34.083 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:34.083 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:34.086 INFO:teuthology.orchestra.run.smithi139.stdout:3916 read 0x77e600 thru 0x77e88e (0x28f bytes) 2024-09-17T23:30:34.090 INFO:teuthology.orchestra.run.smithi139.stdout:3920 trunc from 0x8a5e00 to 0x3b9a800 2024-09-17T23:30:34.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:34.093 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:34.089+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:34.123 INFO:teuthology.orchestra.run.smithi139.stdout:3923 write 0xedc5c00 thru 0xedcf7c8 (0x9bc9 bytes) 2024-09-17T23:30:34.128 INFO:teuthology.orchestra.run.smithi139.stdout:3924 write 0x145bc00 thru 0x1460004 (0x4405 bytes) 2024-09-17T23:30:34.141 INFO:teuthology.orchestra.run.smithi139.stdout:3925 trunc from 0xedcf800 to 0x4a86800 2024-09-17T23:30:34.175 INFO:teuthology.orchestra.run.smithi139.stdout:3928 trunc from 0x4a86800 to 0xdf29600 2024-09-17T23:30:34.186 INFO:teuthology.orchestra.run.smithi139.stdout:3929 trunc from 0xdf29600 to 0xcd40600 2024-09-17T23:30:34.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:34.193 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:34.190+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:34.201 INFO:teuthology.orchestra.run.smithi139.stdout:3931 write 0x3461800 thru 0x347008c (0xe88d bytes) 2024-09-17T23:30:34.205 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:34.205 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:34.206 INFO:teuthology.orchestra.run.smithi139.stdout:3932 punch from 0x9e2b200 to 0x9e39200, (0xe000 bytes) 2024-09-17T23:30:34.207 INFO:teuthology.orchestra.run.smithi139.stdout:3933 write 0x101c800 thru 0x102b5c5 (0xedc6 bytes) 2024-09-17T23:30:34.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:34.221 INFO:teuthology.orchestra.run.smithi139.stdout:3934 trunc from 0xcd40600 to 0x1ae1a00 2024-09-17T23:30:34.241 INFO:teuthology.orchestra.run.smithi139.stdout:3936 trunc from 0x1ae1a00 to 0x8a2600 2024-09-17T23:30:34.263 INFO:teuthology.orchestra.run.smithi139.stdout:3938 read 0x7c4a00 thru 0x7cc490 (0x7a91 bytes) 2024-09-17T23:30:34.283 INFO:teuthology.orchestra.run.smithi139.stdout:3939 write 0x2468600 thru 0x246a82e (0x222f bytes) 2024-09-17T23:30:34.293 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:34.290+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:34.298 INFO:teuthology.orchestra.run.smithi139.stdout:3940 write 0x4ebd000 thru 0x4eccd94 (0xfd95 bytes) 2024-09-17T23:30:34.303 INFO:teuthology.orchestra.run.smithi139.stdout:3941 read 0xd60000 thru 0xd65c25 (0x5c26 bytes) 2024-09-17T23:30:34.304 INFO:teuthology.orchestra.run.smithi139.stdout:3945 read 0xc69400 thru 0xc700b7 (0x6cb8 bytes) 2024-09-17T23:30:34.304 INFO:teuthology.orchestra.run.smithi139.stdout:3946 read 0x1736e00 thru 0x173e8cd (0x7ace bytes) 2024-09-17T23:30:34.305 INFO:teuthology.orchestra.run.smithi139.stdout:3949 read 0x3143e00 thru 0x3147fe5 (0x41e6 bytes) 2024-09-17T23:30:34.305 INFO:teuthology.orchestra.run.smithi139.stdout:3950 read 0x1d2de00 thru 0x1d37a53 (0x9c54 bytes) 2024-09-17T23:30:34.305 INFO:teuthology.orchestra.run.smithi139.stdout:3951 read 0x387a200 thru 0x387ceff (0x2d00 bytes) 2024-09-17T23:30:34.318 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:34.319 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:34.325 INFO:teuthology.orchestra.run.smithi139.stdout:3952 write 0x91dfc00 thru 0x91e5400 (0x5801 bytes) 2024-09-17T23:30:34.336 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:34.344 INFO:teuthology.orchestra.run.smithi139.stdout:3957 write 0xd5a7e00 thru 0xd5b4889 (0xca8a bytes) 2024-09-17T23:30:34.350 INFO:teuthology.orchestra.run.smithi139.stdout:3961 read 0x3452c00 thru 0x345fed2 (0xd2d3 bytes) 2024-09-17T23:30:34.351 INFO:teuthology.orchestra.run.smithi139.stdout:3963 punch from 0x271aa00 to 0x2727a00, (0xd000 bytes) 2024-09-17T23:30:34.352 INFO:teuthology.orchestra.run.smithi139.stdout:3964 write 0xc421600 thru 0xc42f176 (0xdb77 bytes) 2024-09-17T23:30:34.352 INFO:teuthology.orchestra.run.smithi139.stdout:3966 read 0x7d77c00 thru 0x7d86862 (0xec63 bytes) 2024-09-17T23:30:34.352 INFO:teuthology.orchestra.run.smithi139.stdout:3967 read 0x43a5200 thru 0x43b3c6f (0xea70 bytes) 2024-09-17T23:30:34.353 INFO:teuthology.orchestra.run.smithi139.stdout:3968 trunc from 0xd5b4a00 to 0xd6b2600 2024-09-17T23:30:34.353 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:34.354 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:34.365 INFO:teuthology.orchestra.run.smithi139.stdout:3969 write 0x2a52400 thru 0x2a5e160 (0xbd61 bytes) 2024-09-17T23:30:34.366 INFO:teuthology.orchestra.run.smithi139.stdout:3973 write 0x8856400 thru 0x8859333 (0x2f34 bytes) 2024-09-17T23:30:34.366 INFO:teuthology.orchestra.run.smithi139.stdout:3974 write 0x4e9e200 thru 0x4ea10fd (0x2efe bytes) 2024-09-17T23:30:34.370 INFO:teuthology.orchestra.run.smithi139.stdout:3976 read 0x8fa1c00 thru 0x8fb12ea (0xf6eb bytes) 2024-09-17T23:30:34.370 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:34.371 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:34.371 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:34.373 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:34.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:34.384 INFO:teuthology.orchestra.run.smithi139.stdout:3977 write 0xd801800 thru 0xd805be2 (0x43e3 bytes) 2024-09-17T23:30:34.388 INFO:teuthology.orchestra.run.smithi139.stdout:3978 read 0xaa26400 thru 0xaa27884 (0x1485 bytes) 2024-09-17T23:30:34.390 INFO:teuthology.orchestra.run.smithi139.stdout:3979 trunc from 0xd805c00 to 0x846c400 2024-09-17T23:30:34.391 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:34.392 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:34.394 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:34.392+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:34.410 INFO:teuthology.orchestra.run.smithi139.stdout:3980 punch from 0x42a5c00 to 0x42aec00, (0x9000 bytes) 2024-09-17T23:30:34.411 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:34.411 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:34.414 INFO:teuthology.orchestra.run.smithi139.stdout:3981 read 0xda6a00 thru 0xda964f (0x2c50 bytes) 2024-09-17T23:30:34.418 INFO:teuthology.orchestra.run.smithi139.stdout:3983 trunc from 0x846c400 to 0xd13ba00 2024-09-17T23:30:34.435 INFO:teuthology.orchestra.run.smithi139.stdout:3984 read 0x5641800 thru 0x5646f94 (0x5795 bytes) 2024-09-17T23:30:34.435 INFO:teuthology.orchestra.run.smithi139.stdout:3986 write 0xc85800 thru 0xc87f31 (0x2732 bytes) 2024-09-17T23:30:34.435 INFO:teuthology.orchestra.run.smithi139.stdout:3987 write 0x144ea00 thru 0x145c62f (0xdc30 bytes) 2024-09-17T23:30:34.436 INFO:teuthology.orchestra.run.smithi139.stdout:3988 flatten 2024-09-17T23:30:34.461 INFO:teuthology.orchestra.run.smithi139.stdout:3989 trunc from 0xd13ba00 to 0x13d5e00 2024-09-17T23:30:34.478 INFO:teuthology.orchestra.run.smithi139.stdout:3991 read 0xbdc200 thru 0xbdee02 (0x2c03 bytes) 2024-09-17T23:30:34.483 INFO:teuthology.orchestra.run.smithi139.stdout:3992 punch from 0xb49c00 to 0xb4f400, (0x5800 bytes) 2024-09-17T23:30:34.490 INFO:teuthology.orchestra.run.smithi139.stdout:3993 trunc from 0x13d5e00 to 0x722a000 2024-09-17T23:30:34.492 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:34.493 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:34.493 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:34.494 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:34.492+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:34.499 INFO:teuthology.orchestra.run.smithi139.stdout:3995 read 0x1cf7800 thru 0x1d05b42 (0xe343 bytes) 2024-09-17T23:30:34.501 INFO:teuthology.orchestra.run.smithi139.stdout:3997 read 0x4ed9200 thru 0x4ee0e26 (0x7c27 bytes) 2024-09-17T23:30:34.501 INFO:teuthology.orchestra.run.smithi139.stdout:3998 write 0xe96400 thru 0xea4244 (0xde45 bytes) 2024-09-17T23:30:34.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:34.505 INFO:teuthology.orchestra.run.smithi139.stdout:3999 punch from 0x1f3a00 to 0x1f5200, (0x1800 bytes) 2024-09-17T23:30:34.507 INFO:teuthology.orchestra.run.smithi139.stdout:4000 trunc from 0x722a000 to 0x8c87200 2024-09-17T23:30:34.535 INFO:teuthology.orchestra.run.smithi139.stdout:4001 write 0x964d800 thru 0x96560d9 (0x88da bytes) 2024-09-17T23:30:34.536 INFO:teuthology.orchestra.run.smithi139.stdout:4002 punch from 0x294ec00 to 0x2951200, (0x2600 bytes) 2024-09-17T23:30:34.540 INFO:teuthology.orchestra.run.smithi139.stdout:4004 write 0x4182a00 thru 0x418a242 (0x7843 bytes) 2024-09-17T23:30:34.540 INFO:teuthology.orchestra.run.smithi139.stdout:4005 read 0x33e9600 thru 0x33f6961 (0xd362 bytes) 2024-09-17T23:30:34.540 INFO:teuthology.orchestra.run.smithi139.stdout:4006 read 0x6081e00 thru 0x6083f80 (0x2181 bytes) 2024-09-17T23:30:34.540 INFO:teuthology.orchestra.run.smithi139.stdout:4007 trunc from 0x9656200 to 0x9260400 2024-09-17T23:30:34.556 INFO:teuthology.orchestra.run.smithi139.stdout:4009 read 0x8982600 thru 0x8987114 (0x4b15 bytes) 2024-09-17T23:30:34.557 INFO:teuthology.orchestra.run.smithi139.stdout:4010 read 0x88fb400 thru 0x88fbfaf (0xbb0 bytes) 2024-09-17T23:30:34.557 INFO:teuthology.orchestra.run.smithi139.stdout:4011 punch from 0x5b1800 to 0x5b3c00, (0x2400 bytes) 2024-09-17T23:30:34.561 INFO:teuthology.orchestra.run.smithi139.stdout:4016 read 0x1eca600 thru 0x1ed4c2a (0xa62b bytes) 2024-09-17T23:30:34.571 INFO:teuthology.orchestra.run.smithi139.stdout:4018 trunc from 0x9260400 to 0x89cc00 2024-09-17T23:30:34.595 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:34.591+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:34.605 INFO:teuthology.orchestra.run.smithi139.stdout:4019 trunc from 0x89cc00 to 0x89a3200 2024-09-17T23:30:34.614 INFO:teuthology.orchestra.run.smithi139.stdout:4023 punch from 0x493b200 to 0x4941200, (0x6000 bytes) 2024-09-17T23:30:34.616 INFO:teuthology.orchestra.run.smithi139.stdout:4026 read 0x7fe7a00 thru 0x7fe982e (0x1e2f bytes) 2024-09-17T23:30:34.625 INFO:teuthology.orchestra.run.smithi139.stdout:4028 trunc from 0x89a3200 to 0xa6fa00 2024-09-17T23:30:34.656 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.616007185000001, 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:34.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:34.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:34.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.615834245000002, 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:34.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:34.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:34.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:34.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:34.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.615794375, 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:34.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:34.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:34.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.321290148000003, 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:34.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:34.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:34.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 49.073588598999997, 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:34.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.063656332999997, 2024-09-17T23:30:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.050429284000003, 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509900127999998, 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509878737999998, 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509861608999998, 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509844327000003, 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509827297999998, 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509794423999999, 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.509206718000002, 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.487414576999999, 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.486877389, 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.486391272999999, 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.414248670999999, 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.414226859999999, 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.404783989999999, 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:34.785 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:34.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:34.789 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:34.785+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:34.790 INFO:teuthology.orchestra.run.smithi139.stdout:4030 trunc from 0xa6fa00 to 0xeb8da00 2024-09-17T23:30:34.790 INFO:teuthology.orchestra.run.smithi139.stdout:4031 punch from 0x960c400 to 0x961a400, (0xe000 bytes) 2024-09-17T23:30:34.790 INFO:teuthology.orchestra.run.smithi139.stdout:4034 read 0x5889800 thru 0x58973ff (0xdc00 bytes) 2024-09-17T23:30:34.790 INFO:teuthology.orchestra.run.smithi139.stdout:4035 read 0xa94c400 thru 0xa9545b5 (0x81b6 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4039 write 0x9567400 thru 0x9569f60 (0x2b61 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4041 read 0x6992a00 thru 0x69a1e88 (0xf489 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4042 write 0x3e5c00 thru 0x3ea87a (0x4c7b bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4045 punch from 0xad22000 to 0xad25a00, (0x3a00 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4046 write 0x9be2a00 thru 0x9bef9f2 (0xcff3 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4047 punch from 0x673800 to 0x67ce00, (0x9600 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4048 trunc from 0xeb8da00 to 0x355de00 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4050 write 0x2cc0e00 thru 0x2cce920 (0xdb21 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4051 write 0xca42600 thru 0xca454b7 (0x2eb8 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4052 read 0x7429000 thru 0x7438c33 (0xfc34 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4053 read 0xa876a00 thru 0xa87d780 (0x6d81 bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4054 read 0x17c00 thru 0x22fec (0xb3ed bytes) 2024-09-17T23:30:34.791 INFO:teuthology.orchestra.run.smithi139.stdout:4055 read 0xa1d0c00 thru 0xa1dfac4 (0xeec5 bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4056 punch from 0x580da00 to 0x5811600, (0x3c00 bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4057 write 0x4b27800 thru 0x4b2a5ea (0x2deb bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4059 read 0x7a81c00 thru 0x7a8dbf8 (0xbff9 bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4061 read 0x5925000 thru 0x5933f0e (0xef0f bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4062 read 0x839c00 thru 0x840d88 (0x7189 bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4063 read 0x371f800 thru 0x3726354 (0x6b55 bytes) 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4064 trunc from 0xca45600 to 0xfdf800 2024-09-17T23:30:34.792 INFO:teuthology.orchestra.run.smithi139.stdout:4065 punch from 0x15d000 to 0x161e00, (0x4e00 bytes) 2024-09-17T23:30:34.800 INFO:teuthology.orchestra.run.smithi139.stdout:4069 trunc from 0xfdf800 to 0xa65e400 2024-09-17T23:30:34.818 INFO:teuthology.orchestra.run.smithi139.stdout:4071 read 0xa0c4800 thru 0xa0c8d90 (0x4591 bytes) 2024-09-17T23:30:34.818 INFO:teuthology.orchestra.run.smithi139.stdout:4073 write 0x38ce600 thru 0x38d016a (0x1b6b bytes) 2024-09-17T23:30:34.818 INFO:teuthology.orchestra.run.smithi139.stdout:4076 read 0x1bab400 thru 0x1bbacf9 (0xf8fa bytes) 2024-09-17T23:30:34.818 INFO:teuthology.orchestra.run.smithi139.stdout:4080 read 0x27b1600 thru 0x27bf338 (0xdd39 bytes) 2024-09-17T23:30:34.819 INFO:teuthology.orchestra.run.smithi139.stdout:4082 write 0x1383c00 thru 0x1386136 (0x2537 bytes) 2024-09-17T23:30:34.819 INFO:teuthology.orchestra.run.smithi139.stdout:4084 read 0x91c2200 thru 0x91cb226 (0x9027 bytes) 2024-09-17T23:30:34.821 INFO:teuthology.orchestra.run.smithi139.stdout:4087 trunc from 0xa65e400 to 0xcadd600 2024-09-17T23:30:34.835 INFO:teuthology.orchestra.run.smithi139.stdout:4092 read 0xa538200 thru 0xa53dea1 (0x5ca2 bytes) 2024-09-17T23:30:34.836 INFO:teuthology.orchestra.run.smithi139.stdout:4093 read 0xaf62a00 thru 0xaf65069 (0x266a bytes) 2024-09-17T23:30:34.836 INFO:teuthology.orchestra.run.smithi139.stdout:4094 read 0x48c8400 thru 0x48d0b7e (0x877f bytes) 2024-09-17T23:30:34.836 INFO:teuthology.orchestra.run.smithi139.stdout:4095 read 0x1929000 thru 0x19366df (0xd6e0 bytes) 2024-09-17T23:30:34.837 INFO:teuthology.orchestra.run.smithi139.stdout:4097 read 0xa9fa800 thru 0xa9fbb22 (0x1323 bytes) 2024-09-17T23:30:34.841 INFO:teuthology.orchestra.run.smithi139.stdout:4098 trunc from 0xcadd600 to 0x8f05400 2024-09-17T23:30:34.852 INFO:teuthology.orchestra.run.smithi139.stdout:4101 write 0x4e3c200 thru 0x4e4036e (0x416f bytes) 2024-09-17T23:30:34.869 INFO:teuthology.orchestra.run.smithi139.stdout:4102 write 0xaac9a00 thru 0xaacd099 (0x369a bytes) 2024-09-17T23:30:34.873 INFO:teuthology.orchestra.run.smithi139.stdout:4104 trunc from 0xaacd200 to 0xd852800 2024-09-17T23:30:34.885 INFO:teuthology.orchestra.run.smithi139.stdout:4105 write 0x2199c00 thru 0x21a88e4 (0xece5 bytes) 2024-09-17T23:30:34.886 INFO:teuthology.orchestra.run.smithi139.stdout:4106 write 0x3d22000 thru 0x3d2fcf0 (0xdcf1 bytes) 2024-09-17T23:30:34.886 INFO:teuthology.orchestra.run.smithi139.stdout:4108 read 0x7207000 thru 0x7207c13 (0xc14 bytes) 2024-09-17T23:30:34.887 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:34.884+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:34.889 INFO:teuthology.orchestra.run.smithi139.stdout:4109 trunc from 0xd852800 to 0xb156000 2024-09-17T23:30:34.914 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:34.915 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:34.915 INFO:teuthology.orchestra.run.smithi139.stdout:4112 trunc from 0xb156000 to 0x4846c00 2024-09-17T23:30:34.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:34.949 INFO:teuthology.orchestra.run.smithi139.stdout:4113 write 0x7e1a600 thru 0x7e1b561 (0xf62 bytes) 2024-09-17T23:30:34.951 INFO:teuthology.orchestra.run.smithi139.stdout:4115 read 0x5995e00 thru 0x599ab6c (0x4d6d bytes) 2024-09-17T23:30:34.951 INFO:teuthology.orchestra.run.smithi139.stdout:4116 read 0x33ebe00 thru 0x33f54c4 (0x96c5 bytes) 2024-09-17T23:30:34.951 INFO:teuthology.orchestra.run.smithi139.stdout:4117 write 0x2c2bc00 thru 0x2c34ccd (0x90ce bytes) 2024-09-17T23:30:34.951 INFO:teuthology.orchestra.run.smithi139.stdout:4118 write 0x8d7200 thru 0x8e00c7 (0x8ec8 bytes) 2024-09-17T23:30:34.951 INFO:teuthology.orchestra.run.smithi139.stdout:4119 read 0x6898e00 thru 0x689f7f3 (0x69f4 bytes) 2024-09-17T23:30:34.952 INFO:teuthology.orchestra.run.smithi139.stdout:4122 trunc from 0x7e1b600 to 0x6891e00 2024-09-17T23:30:34.980 INFO:teuthology.orchestra.run.smithi139.stdout:4124 trunc from 0x6891e00 to 0x18fc00 2024-09-17T23:30:34.988 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:34.987+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:35.036 INFO:teuthology.orchestra.run.smithi139.stdout:4125 write 0xc9ba400 thru 0xc9bb580 (0x1181 bytes) 2024-09-17T23:30:35.045 INFO:teuthology.orchestra.run.smithi139.stdout:4126 trunc from 0xc9bb600 to 0x5ba6400 2024-09-17T23:30:35.068 INFO:teuthology.orchestra.run.smithi139.stdout:4127 read 0x137a00 thru 0x139b56 (0x2157 bytes) 2024-09-17T23:30:35.088 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:35.089 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:35.089 INFO:teuthology.orchestra.run.smithi139.stdout:4128 write 0x9c6ac00 thru 0x9c79c0f (0xf010 bytes) 2024-09-17T23:30:35.090 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:35.088+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:35.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:35.098 INFO:teuthology.orchestra.run.smithi139.stdout:4129 trunc from 0x9c79e00 to 0x268b800 2024-09-17T23:30:35.123 INFO:teuthology.orchestra.run.smithi139.stdout:4130 read 0x337600 thru 0x344344 (0xcd45 bytes) 2024-09-17T23:30:35.124 INFO:teuthology.orchestra.run.smithi139.stdout:4131 punch from 0x1bd0200 to 0x1bdec00, (0xea00 bytes) 2024-09-17T23:30:35.124 INFO:teuthology.orchestra.run.smithi139.stdout:4133 write 0x112ec00 thru 0x11321c9 (0x35ca bytes) 2024-09-17T23:30:35.124 INFO:teuthology.orchestra.run.smithi139.stdout:4134 punch from 0x1f35600 to 0x1f36400, (0xe00 bytes) 2024-09-17T23:30:35.149 INFO:teuthology.orchestra.run.smithi139.stdout:4137 write 0x5552a00 thru 0x5553791 (0xd92 bytes) 2024-09-17T23:30:35.150 INFO:teuthology.orchestra.run.smithi139.stdout:4139 clone 28 order 23 su 32768 sc 12 2024-09-17T23:30:35.190 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:35.188+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:35.215 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.337201393000001, 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321576262999997, 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321538606999994, 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321521176000005, 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321486245000003, 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321454825000004, 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321438198999999, 2024-09-17T23:30:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321416505000002, 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.321404787000006, 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.362266268000001, 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:30:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.358311104, 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:30:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:30:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.358270446000001, 2024-09-17T23:30:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:30:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.358113711, 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:30:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:30:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.358011020999999, 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:30:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:30:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.357974799000001, 2024-09-17T23:30:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:30:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.357942504, 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:30:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:30:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.357787693000001, 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:30:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:30:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:30:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.3575013729999998, 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:30:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.3574327119999996, 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:30:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.3573907379999994, 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:30:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:30:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.325 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:35.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:35.328 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:35.325+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:35.428 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:35.425+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:35.475 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:35.475 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:35.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:35.529 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:35.527+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:35.629 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:35.626+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:35.631 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:35.632 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:35.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:35.731 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:35.728+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:35.789 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.291493381999999, 2024-09-17T23:30:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.291473502000002, 2024-09-17T23:30:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.291451963999997, 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289569366000002, 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289559250000003, 2024-09-17T23:30:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:35.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289547020000001, 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289536450999996, 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289526563000003, 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289516280999997, 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 58.289500394000001, 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.750575077000001, 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.748505690999998, 2024-09-17T23:30:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.416638323000001, 2024-09-17T23:30:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.931673645, 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:30:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:30:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.931641275, 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.931348898, 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:35.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:30:35.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.92552476, 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:30:35.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.925483257, 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.882744989999999, 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:35.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:30:35.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:30:35.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:35.923 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.882642648999999, 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:35.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:35.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:35.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:30:35.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:35.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.931 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:35.932 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:35.932 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:35.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:30:35.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:35.934 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:35.930+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:36.034 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:36.031+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:36.084 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:36.085 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:36.085 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:36.085 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:36.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:36.135 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:36.133+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:36.235 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:36.234+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:36.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:36.542 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T23:30:36.542 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:36.543 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:36.543 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:36.543 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:36.543 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:36.543 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:36.544 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:36.538+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:36.637 INFO:teuthology.orchestra.run.smithi139.stdout:4142 clone 29 order 25 su 262144 sc 6 2024-09-17T23:30:36.642 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:36.641+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:36.665 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:36.665 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:36.683 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:36.692 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939800390000002, 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939768028000003, 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.698 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:36.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:36.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939746845999998, 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.703 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:36.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:36.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:36.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939720284000003, 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:36.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:36.712 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939698110999998, 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:36.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:36.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939678198999999, 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:36.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:36.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939631523000003, 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:36.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:36.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939604774999999, 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.726 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:36.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939576686999999, 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:36.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:36.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:36.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.939556388, 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:36.736 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:36.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:36.740 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.112757860999999, 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.741 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:36.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3143 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608808191000001, 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0144227690000001, 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.747 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3143 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080796+0000", 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2099999999999995e-07 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080801+0000", 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6909999999999996e-06 2024-09-17T23:30:36.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080822+0000", 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1128000000000002e-05 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080830+0000", 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8280000000000003e-06 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1292999999999997e-05 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3889999999999999e-06 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083687+0000", 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027885470000000002 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083696+0000", 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3959999999999995e-06 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095020+0000", 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011324347 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095106+0000", 2024-09-17T23:30:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5352000000000005e-05 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095218+0000", 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111877 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3144 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608753644, 2024-09-17T23:30:36.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014824425, 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3144 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080850+0000", 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080853+0000", 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2509999999999999e-06 2024-09-17T23:30:36.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080861+0000", 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7840000000000004e-06 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080866+0000", 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080962+0000", 2024-09-17T23:30:36.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5360999999999997e-05 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083738+0000", 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00276998 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083747+0000", 2024-09-17T23:30:36.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1139999999999995e-06 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095514+0000", 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011767071 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095579+0000", 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4732000000000003e-05 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095674+0000", 2024-09-17T23:30:36.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4726000000000001e-05 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3145 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608727780999999, 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014984876, 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3145 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:36.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080876+0000", 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080878+0000", 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-06 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080886+0000", 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3599999999999996e-06 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8559999999999998e-06 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081004+0000", 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112568 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081009+0000", 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083807+0000", 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002797538 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083825+0000", 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8416000000000001e-05 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095733+0000", 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011907194 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095780+0000", 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7942000000000003e-05 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095860+0000", 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9721999999999993e-05 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3146 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608705063999999, 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015105355, 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.765 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3146 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7e-07 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080900+0000", 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8679999999999999e-06 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080906+0000", 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7440000000000004e-06 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080911+0000", 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081059+0000", 2024-09-17T23:30:36.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014868999999999999 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081067+0000", 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5650000000000004e-06 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083866+0000", 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002799008 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083871+0000", 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9509999999999997e-06 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095889+0000", 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012018283 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095931+0000", 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1885999999999998e-05 2024-09-17T23:30:36.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096003+0000", 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2390000000000003e-05 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3147 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608686934000001, 2024-09-17T23:30:36.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015262753, 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3147 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:36.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080917+0000", 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080918+0000", 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7999999999999999e-06 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080924+0000", 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5310000000000001e-06 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080928+0000", 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:30:36.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081109+0000", 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018069200000000001 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081115+0000", 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7490000000000004e-06 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083903+0000", 2024-09-17T23:30:36.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027881249999999998 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083908+0000", 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096026+0000", 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0121184919999999 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096065+0000", 2024-09-17T23:30:36.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8566999999999999e-05 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096179+0000", 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011409900000000001 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3148 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.777 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608650862000001, 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015338839, 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3148 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080953+0000", 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4099999999999999e-07 2024-09-17T23:30:36.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080955+0000", 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.21e-06 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080963+0000", 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7910000000000005e-06 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9350000000000002e-06 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081149+0000", 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018104800000000001 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081152+0000", 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:30:36.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083933+0000", 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027811630000000001 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083937+0000", 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0609999999999997e-06 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096195+0000", 2024-09-17T23:30:36.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122576050000001 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096232+0000", 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7592999999999997e-05 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096291+0000", 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8767999999999997e-05 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3149 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608610294000002, 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154068810000001, 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3149 2024-09-17T23:30:36.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2e-07 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080995+0000", 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1299999999999999e-06 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081003+0000", 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4089999999999997e-06 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081008+0000", 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8940000000000001e-06 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081196+0000", 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018834400000000001 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081202+0000", 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:36.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083961+0000", 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027589340000000002 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083965+0000", 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0029999999999996e-06 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096305+0000", 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012340311 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096343+0000", 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7654000000000002e-05 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096400+0000", 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6876999999999998e-05 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3150 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 28.608589271, 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154925990000001, 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3150 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081016+0000", 2024-09-17T23:30:36.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.916e-06 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081022+0000", 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.66e-06 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081026+0000", 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3490000000000002e-06 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081221+0000", 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000194599 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081223+0000", 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.694e-06 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083982+0000", 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002758122 2024-09-17T23:30:36.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083985+0000", 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9360000000000001e-06 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096413+0000", 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124277079999999 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:36.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096450+0000", 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6449999999999998e-05 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096506+0000", 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6820999999999997e-05 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.796 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4997.0:1730 3.1f 3:fa9e12c1:::rbd_data.1152ce7c866.0000000000000117:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.839092816000001, 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063375800999999995, 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4997", 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1359963294", 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1730 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:36.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850511+0000", 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:36.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850517+0000", 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850522+0000", 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850526+0000", 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913749+0000", 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063222924 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913886+0000", 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013731999999999999 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:36.800 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:36.801 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:36.802 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:36.803 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:36.803 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:36.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:36.804 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:36.804 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:36.804 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:36.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:36.806 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:36.800+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:36.899 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:30:36.902 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:36.901+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:36.959 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:36.959 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:36.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:36.974 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone28 intact 2024-09-17T23:30:37.004 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:37.001+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:37.077 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.077 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:37.129 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:37.128+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:37.194 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.194 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:37.244 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:37.243+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:37.271 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T23:30:37.306 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:37.303+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:37.353 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.353 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:37.407 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:37.404+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:37.429 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:37.429 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:37.429 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 3, 4, 5] 2024-09-17T23:30:37.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-17T23:30:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.473 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:37.508 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:37.506+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:37.536 INFO:teuthology.orchestra.run.smithi139.stdout:4143 read 0x7a9600 thru 0x7aae8f (0x1890 bytes) 2024-09-17T23:30:37.537 INFO:teuthology.orchestra.run.smithi139.stdout:4144 write 0xb68000 thru 0xb69ba3 (0x1ba4 bytes) 2024-09-17T23:30:37.537 INFO:teuthology.orchestra.run.smithi139.stdout:4145 punch from 0x308b600 to 0x3090000, (0x4a00 bytes) 2024-09-17T23:30:37.609 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:37.607+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:37.614 INFO:teuthology.orchestra.run.smithi139.stdout:4149 read 0x179aa00 thru 0x17a798a (0xcf8b bytes) 2024-09-17T23:30:37.614 INFO:teuthology.orchestra.run.smithi139.stdout:4153 read 0x4ade200 thru 0x4ae01b8 (0x1fb9 bytes) 2024-09-17T23:30:37.615 INFO:teuthology.orchestra.run.smithi139.stdout:4154 read 0x27a000 thru 0x27e273 (0x4274 bytes) 2024-09-17T23:30:37.616 INFO:teuthology.orchestra.run.smithi139.stdout:4155 write 0x54e0600 thru 0x54ef075 (0xea76 bytes) 2024-09-17T23:30:37.630 INFO:teuthology.orchestra.run.smithi139.stdout:4157 write 0x6d94800 thru 0x6d94f92 (0x793 bytes) 2024-09-17T23:30:37.630 INFO:teuthology.orchestra.run.smithi139.stdout:4158 read 0x370800 thru 0x37e93b (0xe13c bytes) 2024-09-17T23:30:37.631 INFO:teuthology.orchestra.run.smithi139.stdout:4159 punch from 0x919000 to 0x922a00, (0x9a00 bytes) 2024-09-17T23:30:37.635 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.636 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.637 INFO:teuthology.orchestra.run.smithi139.stdout:4161 read 0x18d8e00 thru 0x18da090 (0x1291 bytes) 2024-09-17T23:30:37.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:37.646 INFO:teuthology.orchestra.run.smithi139.stdout:4162 trunc from 0x6d95000 to 0xd8aba00 2024-09-17T23:30:37.698 INFO:teuthology.orchestra.run.smithi139.stdout:4163 write 0x991c600 thru 0x992c592 (0xff93 bytes) 2024-09-17T23:30:37.710 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:37.707+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:37.716 INFO:teuthology.orchestra.run.smithi139.stdout:4165 trunc from 0xd8aba00 to 0x22aec00 2024-09-17T23:30:37.757 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:30:37.755+0000 7f21c94c5640 -1 mon.a@0(leader).osd e83 definitely_dead 0 2024-09-17T23:30:37.792 INFO:teuthology.orchestra.run.smithi139.stdout:4166 trunc from 0x22aec00 to 0x65a3200 2024-09-17T23:30:37.801 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.801 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.805 INFO:teuthology.orchestra.run.smithi139.stdout:4167 trunc from 0x65a3200 to 0x9f90e00 2024-09-17T23:30:37.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:37.812 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:37.809+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:37.822 INFO:teuthology.orchestra.run.smithi139.stdout:4170 punch from 0x9299000 to 0x92a2e00, (0x9e00 bytes) 2024-09-17T23:30:37.822 INFO:teuthology.orchestra.run.smithi139.stdout:4171 read 0x1724000 thru 0x1724917 (0x918 bytes) 2024-09-17T23:30:37.823 INFO:teuthology.orchestra.run.smithi139.stdout:4172 write 0x5334600 thru 0x533f28b (0xac8c bytes) 2024-09-17T23:30:37.824 INFO:teuthology.orchestra.run.smithi139.stdout:4173 punch from 0xfd7000 to 0xfd7200, (0x200 bytes) 2024-09-17T23:30:37.824 INFO:teuthology.orchestra.run.smithi139.stdout:4174 write 0x8548e00 thru 0x854965b (0x85c bytes) 2024-09-17T23:30:37.824 INFO:teuthology.orchestra.run.smithi139.stdout:4175 punch from 0x2ba8a00 to 0x2bb0e00, (0x8400 bytes) 2024-09-17T23:30:37.824 INFO:teuthology.orchestra.run.smithi139.stdout:4178 read 0x56c4000 thru 0x56cf01f (0xb020 bytes) 2024-09-17T23:30:37.826 INFO:teuthology.orchestra.run.smithi139.stdout:4180 read 0x2406800 thru 0x240ff03 (0x9704 bytes) 2024-09-17T23:30:37.826 INFO:teuthology.orchestra.run.smithi139.stdout:4182 read 0x2fd9000 thru 0x2fdd804 (0x4805 bytes) 2024-09-17T23:30:37.839 INFO:teuthology.orchestra.run.smithi139.stdout:4183 trunc from 0x9f90e00 to 0x563400 2024-09-17T23:30:37.871 INFO:teuthology.orchestra.run.smithi139.stdout:4184 write 0x21ef000 thru 0x21fd33d (0xe33e bytes) 2024-09-17T23:30:37.893 INFO:teuthology.orchestra.run.smithi139.stdout:4186 write 0xb6c9400 thru 0xb6cd334 (0x3f35 bytes) 2024-09-17T23:30:37.894 INFO:teuthology.orchestra.run.smithi139.stdout:4187 read 0x616d000 thru 0x617326d (0x626e bytes) 2024-09-17T23:30:37.895 INFO:teuthology.orchestra.run.smithi139.stdout:4188 trunc from 0xb6cd400 to 0xb9dce00 2024-09-17T23:30:37.906 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.1. 2024-09-17T23:30:37.911 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:37.909+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:37.917 INFO:teuthology.orchestra.run.smithi139.stdout:4189 trunc from 0xb9dce00 to 0x1d82600 2024-09-17T23:30:37.967 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:37.967 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:37.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:38.013 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:38.010+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:38.083 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.083 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:38.137 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:38.134+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:38.201 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.201 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.210 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:38.214 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:38.213+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:38.315 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:38.314+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:38.357 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.357 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.365 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:38.416 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:38.413+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:38.522 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:38.519+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:38.618 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:38.616+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:38.628 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.628 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:38.718 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:38.715+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:38.743 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.743 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:38.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:38.837 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:38.834+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:38.900 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:38.901 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:38.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:38.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:30:38.949 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:38.949 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:38.951 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:38.948+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:38.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:38.967 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:38.983 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:38.983 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:39.001 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:39.001 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:39.018 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:39.023 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:39.021+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:39.037 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:39.059 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:39.059 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:39.059 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:39.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:39.124 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:39.121+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:39.174 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:39.174 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:39.175 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:39.175 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:39.183 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:39.225 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:39.223+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:39.325 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:39.322+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:39.332 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579950472999997, 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.333 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:39.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.337 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579918110999998, 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:39.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:39.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579896929, 2024-09-17T23:30:39.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:39.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579870366999998, 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:39.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.351 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579848194, 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:39.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:39.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579828282000001, 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.356 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:39.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.360 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579781605999997, 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:39.364 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579754858000001, 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.365 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:39.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:39.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.369 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579726770000001, 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:39.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.579706471000001, 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:39.374 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:39.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.378 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 45.752907944, 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:39.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:39.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:39.384 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3143 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248958274, 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0144227690000001, 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3143 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080796+0000", 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2099999999999995e-07 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080801+0000", 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6909999999999996e-06 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080822+0000", 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1128000000000002e-05 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080830+0000", 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8280000000000003e-06 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1292999999999997e-05 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3889999999999999e-06 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083687+0000", 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027885470000000002 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083696+0000", 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3959999999999995e-06 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095020+0000", 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011324347 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095106+0000", 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5352000000000005e-05 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095218+0000", 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111877 2024-09-17T23:30:39.390 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3144 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248903726999998, 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014824425, 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.391 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3144 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080850+0000", 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080853+0000", 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2509999999999999e-06 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080861+0000", 2024-09-17T23:30:39.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7840000000000004e-06 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080866+0000", 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080962+0000", 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5360999999999997e-05 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083738+0000", 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00276998 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083747+0000", 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1139999999999995e-06 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095514+0000", 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011767071 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095579+0000", 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4732000000000003e-05 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095674+0000", 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4726000000000001e-05 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.396 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3145 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248877864000001, 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014984876, 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.397 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3145 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080876+0000", 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080878+0000", 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-06 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080886+0000", 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3599999999999996e-06 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8559999999999998e-06 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081004+0000", 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112568 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081009+0000", 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083807+0000", 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002797538 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083825+0000", 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8416000000000001e-05 2024-09-17T23:30:39.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095733+0000", 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011907194 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095780+0000", 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7942000000000003e-05 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095860+0000", 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9721999999999993e-05 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3146 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248855147, 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015105355, 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.403 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3146 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:39.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7e-07 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080900+0000", 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8679999999999999e-06 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080906+0000", 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7440000000000004e-06 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080911+0000", 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081059+0000", 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014868999999999999 2024-09-17T23:30:39.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081067+0000", 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5650000000000004e-06 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083866+0000", 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002799008 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083871+0000", 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9509999999999997e-06 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095889+0000", 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012018283 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095931+0000", 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1885999999999998e-05 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096003+0000", 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2390000000000003e-05 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3147 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248837017, 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015262753, 2024-09-17T23:30:39.409 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3147 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080917+0000", 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080918+0000", 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7999999999999999e-06 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080924+0000", 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5310000000000001e-06 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080928+0000", 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081109+0000", 2024-09-17T23:30:39.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018069200000000001 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081115+0000", 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7490000000000004e-06 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083903+0000", 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027881249999999998 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083908+0000", 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096026+0000", 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0121184919999999 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096065+0000", 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8566999999999999e-05 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096179+0000", 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011409900000000001 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3148 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248800944999999, 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015338839, 2024-09-17T23:30:39.415 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3148 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080953+0000", 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4099999999999999e-07 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080955+0000", 2024-09-17T23:30:39.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.21e-06 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080963+0000", 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7910000000000005e-06 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9350000000000002e-06 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081149+0000", 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018104800000000001 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081152+0000", 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083933+0000", 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027811630000000001 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083937+0000", 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0609999999999997e-06 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096195+0000", 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122576050000001 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096232+0000", 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7592999999999997e-05 2024-09-17T23:30:39.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096291+0000", 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8767999999999997e-05 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3149 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248760377, 2024-09-17T23:30:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154068810000001, 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3149 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:39.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2e-07 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080995+0000", 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1299999999999999e-06 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081003+0000", 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4089999999999997e-06 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081008+0000", 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8940000000000001e-06 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081196+0000", 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018834400000000001 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081202+0000", 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083961+0000", 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027589340000000002 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083965+0000", 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0029999999999996e-06 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096305+0000", 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012340311 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096343+0000", 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7654000000000002e-05 2024-09-17T23:30:39.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096400+0000", 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6876999999999998e-05 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3150 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.248739354000001, 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154925990000001, 2024-09-17T23:30:39.427 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3150 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081016+0000", 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.916e-06 2024-09-17T23:30:39.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081022+0000", 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.66e-06 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081026+0000", 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3490000000000002e-06 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081221+0000", 2024-09-17T23:30:39.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000194599 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081223+0000", 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.694e-06 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083982+0000", 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002758122 2024-09-17T23:30:39.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083985+0000", 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9360000000000001e-06 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096413+0000", 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124277079999999 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:39.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096450+0000", 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6449999999999998e-05 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096506+0000", 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6820999999999997e-05 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.433 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4997.0:1730 3.1f 3:fa9e12c1:::rbd_data.1152ce7c866.0000000000000117:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-17T23:30:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.479242898999999, 2024-09-17T23:30:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063375800999999995, 2024-09-17T23:30:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:39.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4997", 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1359963294", 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1730 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:39.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850511+0000", 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850517+0000", 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850522+0000", 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850526+0000", 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913749+0000", 2024-09-17T23:30:39.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063222924 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913886+0000", 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013731999999999999 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:39.440 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:39.442 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:39.439+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:39.442 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:39.542 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:39.540+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:39.551 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:39.552 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:39.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:39.644 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:39.641+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:39.707 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:39.708 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:39.716 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:39.745 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:39.742+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.980182430999999, 2024-09-17T23:30:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:30:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.970610332, 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:30:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.967079825999999, 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:30:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.96558512, 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:30:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.965402201, 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:30:39.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:30:39.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.965288699999999, 2024-09-17T23:30:39.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:30:39.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:30:39.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.965264593000001, 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:39.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:30:39.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.965190262, 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:30:39.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:30:39.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.965166712999999, 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:30:39.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:30:39.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.961995512000001, 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:30:39.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:30:39.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:39.876 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.957587176000001, 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:30:39.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.930315068999999, 2024-09-17T23:30:39.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:39.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:30:39.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:30:39.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.921132917000001, 2024-09-17T23:30:39.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:30:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:30:39.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.921101424, 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:30:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.921075731999998, 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:30:39.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:30:39.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:30:39.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.92093118, 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:30:39.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.919363062999999, 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:39.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.919165715999998, 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:30:39.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:30:39.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:39.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.919139400999999, 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:39.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:30:39.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:39.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.919111057999999, 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:39.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:39.931 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:39.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:30:39.934 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:39.930+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:39.967 INFO:teuthology.orchestra.run.smithi139.stdout:4192 read 0x11b1c00 thru 0x11b4ca0 (0x30a1 bytes) 2024-09-17T23:30:39.990 INFO:teuthology.orchestra.run.smithi139.stdout:4196 write 0x3f66800 thru 0x3f6e57d (0x7d7e bytes) 2024-09-17T23:30:39.992 INFO:teuthology.orchestra.run.smithi139.stdout:4197 read 0x689c00 thru 0x691eaf (0x82b0 bytes) 2024-09-17T23:30:39.992 INFO:teuthology.orchestra.run.smithi139.stdout:4198 read 0x3925e00 thru 0x393410d (0xe30e bytes) 2024-09-17T23:30:40.034 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:40.032+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:40.034 INFO:teuthology.orchestra.run.smithi139.stdout:4199 write 0xd17ce00 thru 0xd18a17d (0xd37e bytes) 2024-09-17T23:30:40.036 INFO:teuthology.orchestra.run.smithi139.stdout:4200 read 0x13bda00 thru 0x13c76a0 (0x9ca1 bytes) 2024-09-17T23:30:40.037 INFO:teuthology.orchestra.run.smithi139.stdout:4202 write 0xcd7600 thru 0xcdd73e (0x613f bytes) 2024-09-17T23:30:40.038 INFO:teuthology.orchestra.run.smithi139.stdout:4205 write 0x44fea00 thru 0x4505b31 (0x7132 bytes) 2024-09-17T23:30:40.039 INFO:teuthology.orchestra.run.smithi139.stdout:4209 read 0x31f5400 thru 0x320235a (0xcf5b bytes) 2024-09-17T23:30:40.043 INFO:teuthology.orchestra.run.smithi139.stdout:4212 trunc from 0xd18a200 to 0xabe2a00 2024-09-17T23:30:40.089 INFO:teuthology.orchestra.run.smithi139.stdout:4213 read 0xa56d200 thru 0xa56ee80 (0x1c81 bytes) 2024-09-17T23:30:40.090 INFO:teuthology.orchestra.run.smithi139.stdout:4214 write 0x8c58a00 thru 0x8c67483 (0xea84 bytes) 2024-09-17T23:30:40.094 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:40.094 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:40.094 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:40.094 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:40.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:40.121 INFO:teuthology.orchestra.run.smithi139.stdout:4215 write 0xe20b000 thru 0xe2104f1 (0x54f2 bytes) 2024-09-17T23:30:40.123 INFO:teuthology.orchestra.run.smithi139.stdout:4217 read 0xe60400 thru 0xe68e61 (0x8a62 bytes) 2024-09-17T23:30:40.123 INFO:teuthology.orchestra.run.smithi139.stdout:4220 write 0x99aa200 thru 0x99b6952 (0xc753 bytes) 2024-09-17T23:30:40.124 INFO:teuthology.orchestra.run.smithi139.stdout:4221 punch from 0x5eb7a00 to 0x5ec3a00, (0xc000 bytes) 2024-09-17T23:30:40.125 INFO:teuthology.orchestra.run.smithi139.stdout:4223 write 0x966a00 thru 0x96fc5e (0x925f bytes) 2024-09-17T23:30:40.129 INFO:teuthology.orchestra.run.smithi139.stdout:4226 punch from 0x3857e00 to 0x385d600, (0x5800 bytes) 2024-09-17T23:30:40.147 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:40.145+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:40.148 INFO:teuthology.orchestra.run.smithi139.stdout:4227 trunc from 0xe210600 to 0x55a400 2024-09-17T23:30:40.192 INFO:teuthology.orchestra.run.smithi139.stdout:4229 trunc from 0x55a400 to 0xceacc00 2024-09-17T23:30:40.204 INFO:teuthology.orchestra.run.smithi139.stdout:4230 read 0x290da00 thru 0x2914d5c (0x735d bytes) 2024-09-17T23:30:40.205 INFO:teuthology.orchestra.run.smithi139.stdout:4231 read 0x7a80000 thru 0x7a81cb0 (0x1cb1 bytes) 2024-09-17T23:30:40.205 INFO:teuthology.orchestra.run.smithi139.stdout:4232 write 0x5a38400 thru 0x5a38c83 (0x884 bytes) 2024-09-17T23:30:40.206 INFO:teuthology.orchestra.run.smithi139.stdout:4235 read 0x2c0ec00 thru 0x2c175f4 (0x89f5 bytes) 2024-09-17T23:30:40.206 INFO:teuthology.orchestra.run.smithi139.stdout:4237 read 0x6be2400 thru 0x6bf07c1 (0xe3c2 bytes) 2024-09-17T23:30:40.222 INFO:teuthology.orchestra.run.smithi139.stdout:4241 write 0xe6b5e00 thru 0xe6be996 (0x8b97 bytes) 2024-09-17T23:30:40.224 INFO:teuthology.orchestra.run.smithi139.stdout:4242 read 0x4c2600 thru 0x4ca70a (0x810b bytes) 2024-09-17T23:30:40.224 INFO:teuthology.orchestra.run.smithi139.stdout:4243 read 0xaba4e00 thru 0xabab303 (0x6504 bytes) 2024-09-17T23:30:40.224 INFO:teuthology.orchestra.run.smithi139.stdout:4244 read 0x2498400 thru 0x249bc4a (0x384b bytes) 2024-09-17T23:30:40.225 INFO:teuthology.orchestra.run.smithi139.stdout:4245 read 0x4908a00 thru 0x49185dc (0xfbdd bytes) 2024-09-17T23:30:40.225 INFO:teuthology.orchestra.run.smithi139.stdout:4247 write 0x12dec00 thru 0x12e51f3 (0x65f4 bytes) 2024-09-17T23:30:40.226 INFO:teuthology.orchestra.run.smithi139.stdout:4251 punch from 0x32aba00 to 0x32af000, (0x3600 bytes) 2024-09-17T23:30:40.227 INFO:teuthology.orchestra.run.smithi139.stdout:4252 trunc from 0xe6bea00 to 0xe2b6a00 2024-09-17T23:30:40.236 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:40.233+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:40.245 INFO:teuthology.orchestra.run.smithi139.stdout:4253 read 0x28d1800 thru 0x28d92cf (0x7ad0 bytes) 2024-09-17T23:30:40.246 INFO:teuthology.orchestra.run.smithi139.stdout:4254 read 0x9d79000 thru 0x9d7ecb4 (0x5cb5 bytes) 2024-09-17T23:30:40.246 INFO:teuthology.orchestra.run.smithi139.stdout:4255 write 0x6442000 thru 0x644ae39 (0x8e3a bytes) 2024-09-17T23:30:40.247 INFO:teuthology.orchestra.run.smithi139.stdout:4256 write 0x8a19e00 thru 0x8a1e5d1 (0x47d2 bytes) 2024-09-17T23:30:40.248 INFO:teuthology.orchestra.run.smithi139.stdout:4257 write 0x3ef5600 thru 0x3ef63e1 (0xde2 bytes) 2024-09-17T23:30:40.249 INFO:teuthology.orchestra.run.smithi139.stdout:4259 punch from 0x96c8e00 to 0x96d2600, (0x9800 bytes) 2024-09-17T23:30:40.262 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:40.262 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:40.263 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:40.263 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:40.263 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:40.264 INFO:teuthology.orchestra.run.smithi139.stdout:4262 trunc from 0xe2b6a00 to 0xe48e00 2024-09-17T23:30:40.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:40.335 INFO:teuthology.orchestra.run.smithi139.stdout:4264 write 0x6d41e00 thru 0x6d4f4a7 (0xd6a8 bytes) 2024-09-17T23:30:40.336 INFO:teuthology.orchestra.run.smithi139.stdout:4265 write 0x487d600 thru 0x487ef63 (0x1964 bytes) 2024-09-17T23:30:40.337 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:40.335+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:40.337 INFO:teuthology.orchestra.run.smithi139.stdout:4267 read 0x254cc00 thru 0x254e3cf (0x17d0 bytes) 2024-09-17T23:30:40.338 INFO:teuthology.orchestra.run.smithi139.stdout:4268 read 0x1438c00 thru 0x143d46c (0x486d bytes) 2024-09-17T23:30:40.338 INFO:teuthology.orchestra.run.smithi139.stdout:4271 read 0x3f66200 thru 0x3f701c5 (0x9fc6 bytes) 2024-09-17T23:30:40.339 INFO:teuthology.orchestra.run.smithi139.stdout:4273 punch from 0x2e1d400 to 0x2e28e00, (0xba00 bytes) 2024-09-17T23:30:40.339 INFO:teuthology.orchestra.run.smithi139.stdout:4277 write 0x63cfa00 thru 0x63d2cf4 (0x32f5 bytes) 2024-09-17T23:30:40.361 INFO:teuthology.orchestra.run.smithi139.stdout:4279 write 0xe463e00 thru 0xe46fdbd (0xbfbe bytes) 2024-09-17T23:30:40.362 INFO:teuthology.orchestra.run.smithi139.stdout:4280 write 0xa6e000 thru 0xa71cf6 (0x3cf7 bytes) 2024-09-17T23:30:40.363 INFO:teuthology.orchestra.run.smithi139.stdout:4282 write 0x8d9a200 thru 0x8d9a6de (0x4df bytes) 2024-09-17T23:30:40.363 INFO:teuthology.orchestra.run.smithi139.stdout:4283 read 0x6c18800 thru 0x6c1d65c (0x4e5d bytes) 2024-09-17T23:30:40.367 INFO:teuthology.orchestra.run.smithi139.stdout:4284 trunc from 0xe46fe00 to 0xb542800 2024-09-17T23:30:40.385 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.911320508, 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.053484695000002, 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.052792402000001, 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.052780820999999, 2024-09-17T23:30:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.052253573999998, 2024-09-17T23:30:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.052219590999997, 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.051908998999998, 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.051882865000003, 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.051384958, 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.050783868000003, 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.050235233000002, 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.050027634999999, 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.040411734000003, 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:40.443 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:40.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:40.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.040401500000002, 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:40.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:40.452 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.040391632999999, 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:40.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:40.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.040381484999997, 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:40.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.039911617000001, 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.039889500999998, 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:40.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:40.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.039601806, 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.471 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:40.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.854215635999999, 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:40.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:40.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.481 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:40.483 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:40.479+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:40.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4286 write 0x2f2c800 thru 0x2f3c753 (0xff54 bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4288 read 0x753b600 thru 0x753dcea (0x26eb bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4289 punch from 0x81c200 to 0x829200, (0xd000 bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4291 punch from 0x5323e00 to 0x5327400, (0x3600 bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4293 punch from 0x66afa00 to 0x66b7a00, (0x8000 bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4294 read 0x80b8e00 thru 0x80b9fad (0x11ae bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4295 read 0x2996400 thru 0x299aa56 (0x4657 bytes) 2024-09-17T23:30:40.485 INFO:teuthology.orchestra.run.smithi139.stdout:4296 write 0x6850400 thru 0x685f377 (0xef78 bytes) 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4297 read 0x46c0400 thru 0x46c901b (0x8c1c bytes) 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4300 trunc from 0xb542800 to 0x318c600 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4302 trunc from 0x318c600 to 0x8a0d600 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4306 write 0x87fa00 thru 0x886ebd (0x74be bytes) 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4307 punch from 0x307f800 to 0x308aa00, (0xb200 bytes) 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4308 read 0x2a89200 thru 0x2a9892e (0xf72f bytes) 2024-09-17T23:30:40.486 INFO:teuthology.orchestra.run.smithi139.stdout:4309 trunc from 0x8a0d600 to 0x4f3d800 2024-09-17T23:30:40.583 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:40.581+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:40.633 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:40.633 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:40.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:40.684 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:40.682+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:40.688 INFO:teuthology.orchestra.run.smithi139.stdout:4313 write 0x225ae00 thru 0x225de17 (0x3018 bytes) 2024-09-17T23:30:40.690 INFO:teuthology.orchestra.run.smithi139.stdout:4314 write 0x4b6400 thru 0x4b6bc0 (0x7c1 bytes) 2024-09-17T23:30:40.700 INFO:teuthology.orchestra.run.smithi139.stdout:4315 trunc from 0x4f3d800 to 0xc4e6200 2024-09-17T23:30:40.721 INFO:teuthology.orchestra.run.smithi139.stdout:4316 write 0xd7b1200 thru 0xd7bb867 (0xa668 bytes) 2024-09-17T23:30:40.722 INFO:teuthology.orchestra.run.smithi139.stdout:4317 write 0x497c00 thru 0x49ea34 (0x6e35 bytes) 2024-09-17T23:30:40.733 INFO:teuthology.orchestra.run.smithi139.stdout:4318 trunc from 0xd7bba00 to 0x55a3600 2024-09-17T23:30:40.758 INFO:teuthology.orchestra.run.smithi139.stdout:4321 punch from 0x2913800 to 0x2916200, (0x2a00 bytes) 2024-09-17T23:30:40.759 INFO:teuthology.orchestra.run.smithi139.stdout:4322 punch from 0x3cf6800 to 0x3d03a00, (0xd200 bytes) 2024-09-17T23:30:40.759 INFO:teuthology.orchestra.run.smithi139.stdout:4324 read 0x4151e00 thru 0x4153c61 (0x1e62 bytes) 2024-09-17T23:30:40.761 INFO:teuthology.orchestra.run.smithi139.stdout:4325 read 0x1b47e00 thru 0x1b545a1 (0xc7a2 bytes) 2024-09-17T23:30:40.767 INFO:teuthology.orchestra.run.smithi139.stdout:4327 trunc from 0x55a3600 to 0xaac2c00 2024-09-17T23:30:40.784 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:40.781+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:40.792 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:40.792 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:40.793 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:40.793 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:40.794 INFO:teuthology.orchestra.run.smithi139.stdout:4328 trunc from 0xaac2c00 to 0xa12400 2024-09-17T23:30:40.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:40.833 INFO:teuthology.orchestra.run.smithi139.stdout:4329 trunc from 0xa12400 to 0xde45c00 2024-09-17T23:30:40.847 INFO:teuthology.orchestra.run.smithi139.stdout:4331 read 0x43b8000 thru 0x43c2095 (0xa096 bytes) 2024-09-17T23:30:40.848 INFO:teuthology.orchestra.run.smithi139.stdout:4333 punch from 0x8e72600 to 0x8e72a00, (0x400 bytes) 2024-09-17T23:30:40.886 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:40.883+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:40.916 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:40.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.877654565, 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:40.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:40.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.877481625000001, 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:40.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:40.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:40.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.877441755, 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:40.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:40.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.582937528000002, 2024-09-17T23:30:40.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:40.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.335235978999997, 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:40.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:40.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:40.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.325303712999997, 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:40.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.312076664000003, 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:40.949 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:40.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:40.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.771547507999998, 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:40.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.771526117999997, 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:40.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.771508988999997, 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.771491707000003, 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.771474677999997, 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.771441803999998, 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.770854098000001, 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.749061957000002, 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.748524768999999, 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.748038653000002, 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675896050999999, 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675874239999999, 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.666431370000002, 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:41.020 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:41.022 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:41.022 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:41.019+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:41.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:41.123 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:41.120+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:41.130 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:41.130 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:41.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.130 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5437.0:2525 3.11 3.8811e1b1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:40.847555+0000", 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.28159026300000001, 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.28161013499999998, 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5437", 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1040410328", 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2525 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:41.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847555+0000", 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847555+0000", 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847555+0000", 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7199999999999999e-07 2024-09-17T23:30:41.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847558+0000", 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0800000000000002e-06 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847564+0000", 2024-09-17T23:30:41.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6640000000000003e-06 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847568+0000", 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8449999999999996e-06 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847621+0000", 2024-09-17T23:30:41.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3301999999999999e-05 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.847630+0000", 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.969e-06 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.849052+0000", 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014223409999999999 2024-09-17T23:30:41.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:40.849061+0000", 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1859999999999992e-06 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 1 2024-09-17T23:30:41.136 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:41.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:41.184 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:41.184 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:41.201 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:41.217 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:41.217 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:41.224 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:41.221+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:41.234 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:41.234 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:41.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:41.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:41.268 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:41.288 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:41.288 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:41.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:41.338 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:41.335+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:41.433 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:41.430+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:41.444 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:41.445 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:41.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:41.527 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:41.524+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:41.561 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:41.561 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:41.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:41.628 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:41.625+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:41.676 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:41.676 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:41.685 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:41.729 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:41.726+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:41.830 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:41.827+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:41.834 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:41.834 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:41.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:41.882 INFO:teuthology.orchestra.run.smithi139.stdout:4334 write 0xe691000 thru 0xe695ca2 (0x4ca3 bytes) 2024-09-17T23:30:41.882 INFO:teuthology.orchestra.run.smithi139.stdout:4335 write 0x9040200 thru 0x904dce1 (0xdae2 bytes) 2024-09-17T23:30:41.883 INFO:teuthology.orchestra.run.smithi139.stdout:4336 read 0xc0e600 thru 0xc16e25 (0x8826 bytes) 2024-09-17T23:30:41.883 INFO:teuthology.orchestra.run.smithi139.stdout:4337 punch from 0x9a8f800 to 0x9a9da00, (0xe200 bytes) 2024-09-17T23:30:41.884 INFO:teuthology.orchestra.run.smithi139.stdout:4338 read 0x99ae800 thru 0x99baeea (0xc6eb bytes) 2024-09-17T23:30:41.886 INFO:teuthology.orchestra.run.smithi139.stdout:4340 write 0xae08600 thru 0xae13952 (0xb353 bytes) 2024-09-17T23:30:41.886 INFO:teuthology.orchestra.run.smithi139.stdout:4341 punch from 0x8556000 to 0x8559000, (0x3000 bytes) 2024-09-17T23:30:41.895 INFO:teuthology.orchestra.run.smithi139.stdout:4343 trunc from 0xe695e00 to 0x9a50000 2024-09-17T23:30:41.924 INFO:teuthology.orchestra.run.smithi139.stdout:4345 punch from 0x56fc600 to 0x56ff000, (0x2a00 bytes) 2024-09-17T23:30:41.925 INFO:teuthology.orchestra.run.smithi139.stdout:4346 punch from 0x3a70800 to 0x3a72e00, (0x2600 bytes) 2024-09-17T23:30:41.931 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:41.928+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:41.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:30:41.938 INFO:teuthology.orchestra.run.smithi139.stdout:4347 trunc from 0x9a50000 to 0x10f2e00 2024-09-17T23:30:41.967 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:41.968 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:41.977 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:41.989 INFO:teuthology.orchestra.run.smithi139.stdout:4348 write 0x6f9d200 thru 0x6fa5cde (0x8adf bytes) 2024-09-17T23:30:42.007 INFO:teuthology.orchestra.run.smithi139.stdout:4349 write 0xbf2e800 thru 0xbf3c95f (0xe160 bytes) 2024-09-17T23:30:42.011 INFO:teuthology.orchestra.run.smithi139.stdout:4350 trunc from 0xbf3ca00 to 0xe119a00 2024-09-17T23:30:42.022 INFO:teuthology.orchestra.run.smithi139.stdout:4352 write 0xc085000 thru 0xc08756a (0x256b bytes) 2024-09-17T23:30:42.022 INFO:teuthology.orchestra.run.smithi139.stdout:4353 read 0x6660c00 thru 0x666e790 (0xdb91 bytes) 2024-09-17T23:30:42.023 INFO:teuthology.orchestra.run.smithi139.stdout:4354 punch from 0x109de00 to 0x10aca00, (0xec00 bytes) 2024-09-17T23:30:42.023 INFO:teuthology.orchestra.run.smithi139.stdout:4357 write 0x624d000 thru 0x625ca3a (0xfa3b bytes) 2024-09-17T23:30:42.024 INFO:teuthology.orchestra.run.smithi139.stdout:4358 read 0xddfa800 thru 0xddffa43 (0x5244 bytes) 2024-09-17T23:30:42.024 INFO:teuthology.orchestra.run.smithi139.stdout:4359 read 0x1ed4c00 thru 0x1eda7e0 (0x5be1 bytes) 2024-09-17T23:30:42.024 INFO:teuthology.orchestra.run.smithi139.stdout:4360 write 0xe6de00 thru 0xe7d226 (0xf427 bytes) 2024-09-17T23:30:42.025 INFO:teuthology.orchestra.run.smithi139.stdout:4361 write 0x98ce400 thru 0x98d0763 (0x2364 bytes) 2024-09-17T23:30:42.026 INFO:teuthology.orchestra.run.smithi139.stdout:4362 write 0x21d3c00 thru 0x21d52da (0x16db bytes) 2024-09-17T23:30:42.027 INFO:teuthology.orchestra.run.smithi139.stdout:4366 write 0x6e3e400 thru 0x6e456fe (0x72ff bytes) 2024-09-17T23:30:42.027 INFO:teuthology.orchestra.run.smithi139.stdout:4367 write 0x3ef4800 thru 0x3ef74fd (0x2cfe bytes) 2024-09-17T23:30:42.028 INFO:teuthology.orchestra.run.smithi139.stdout:4368 punch from 0x8211c00 to 0x821cc00, (0xb000 bytes) 2024-09-17T23:30:42.028 INFO:teuthology.orchestra.run.smithi139.stdout:4371 write 0x2d8c00 thru 0x2e7bd1 (0xefd2 bytes) 2024-09-17T23:30:42.032 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:42.029+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:42.049 INFO:teuthology.orchestra.run.smithi139.stdout:4373 write 0xe593200 thru 0xe595978 (0x2779 bytes) 2024-09-17T23:30:42.050 INFO:teuthology.orchestra.run.smithi139.stdout:4376 write 0x5ab9000 thru 0x5ac845f (0xf460 bytes) 2024-09-17T23:30:42.050 INFO:teuthology.orchestra.run.smithi139.stdout:4378 write 0x5136400 thru 0x51398d3 (0x34d4 bytes) 2024-09-17T23:30:42.052 INFO:teuthology.orchestra.run.smithi139.stdout:4381 trunc from 0xe595a00 to 0xd22bc00 2024-09-17T23:30:42.080 INFO:teuthology.orchestra.run.smithi139.stdout:4382 punch from 0xba6000 to 0xbabc00, (0x5c00 bytes) 2024-09-17T23:30:42.080 INFO:teuthology.orchestra.run.smithi139.stdout:4383 write 0x4d4be00 thru 0x4d4ee8e (0x308f bytes) 2024-09-17T23:30:42.080 INFO:teuthology.orchestra.run.smithi139.stdout:4385 write 0x6045800 thru 0x6046810 (0x1011 bytes) 2024-09-17T23:30:42.081 INFO:teuthology.orchestra.run.smithi139.stdout:4386 read 0xafd3200 thru 0xafdeeca (0xbccb bytes) 2024-09-17T23:30:42.084 INFO:teuthology.orchestra.run.smithi139.stdout:4389 trunc from 0xd22bc00 to 0xa50ec00 2024-09-17T23:30:42.100 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.100 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.108 INFO:teuthology.orchestra.run.smithi139.stdout:4390 write 0x2427a00 thru 0x2431e57 (0xa458 bytes) 2024-09-17T23:30:42.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:42.126 INFO:teuthology.orchestra.run.smithi139.stdout:4391 write 0xe0e5800 thru 0xe0e8601 (0x2e02 bytes) 2024-09-17T23:30:42.127 INFO:teuthology.orchestra.run.smithi139.stdout:4392 read 0xd611c00 thru 0xd61f284 (0xd685 bytes) 2024-09-17T23:30:42.127 INFO:teuthology.orchestra.run.smithi139.stdout:4395 punch from 0xd1bb800 to 0xd1bc400, (0xc00 bytes) 2024-09-17T23:30:42.128 INFO:teuthology.orchestra.run.smithi139.stdout:4397 trunc from 0xe0e8800 to 0xd761600 2024-09-17T23:30:42.145 INFO:teuthology.orchestra.run.smithi139.stdout:4399 write 0x75c000 thru 0x75e834 (0x2835 bytes) 2024-09-17T23:30:42.155 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:42.152+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:42.161 INFO:teuthology.orchestra.run.smithi139.stdout:4401 trunc from 0xd761600 to 0xcaba00 2024-09-17T23:30:42.189 INFO:teuthology.orchestra.run.smithi139.stdout:4402 trunc from 0xcaba00 to 0x2c58600 2024-09-17T23:30:42.198 INFO:teuthology.orchestra.run.smithi139.stdout:4403 write 0x16dac00 thru 0x16dd991 (0x2d92 bytes) 2024-09-17T23:30:42.227 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.227 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.229 INFO:teuthology.orchestra.run.smithi139.stdout:4404 write 0xd2ca800 thru 0xd2d039a (0x5b9b bytes) 2024-09-17T23:30:42.233 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:42.231+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:4409 trunc from 0xd2d0400 to 0x899ba00 2024-09-17T23:30:42.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:42.271 INFO:teuthology.orchestra.run.smithi139.stdout:4412 write 0x8e6e400 thru 0x8e773e7 (0x8fe8 bytes) 2024-09-17T23:30:42.276 INFO:teuthology.orchestra.run.smithi139.stdout:4413 punch from 0x58e1000 to 0x58eb200, (0xa200 bytes) 2024-09-17T23:30:42.276 INFO:teuthology.orchestra.run.smithi139.stdout:4414 write 0x6ae000 thru 0x6b8bc0 (0xabc1 bytes) 2024-09-17T23:30:42.277 INFO:teuthology.orchestra.run.smithi139.stdout:4415 punch from 0x7a02c00 to 0x7a04200, (0x1600 bytes) 2024-09-17T23:30:42.277 INFO:teuthology.orchestra.run.smithi139.stdout:4416 read 0x221e400 thru 0x2229128 (0xad29 bytes) 2024-09-17T23:30:42.277 INFO:teuthology.orchestra.run.smithi139.stdout:4417 punch from 0x4b8d600 to 0x4b94c00, (0x7600 bytes) 2024-09-17T23:30:42.277 INFO:teuthology.orchestra.run.smithi139.stdout:4419 write 0x88af600 thru 0x88b3201 (0x3c02 bytes) 2024-09-17T23:30:42.292 INFO:teuthology.orchestra.run.smithi139.stdout:4420 write 0xb1aec00 thru 0xb1b90c1 (0xa4c2 bytes) 2024-09-17T23:30:42.301 INFO:teuthology.orchestra.run.smithi139.stdout:4425 trunc from 0xb1b9200 to 0x8bfe200 2024-09-17T23:30:42.319 INFO:teuthology.orchestra.run.smithi139.stdout:4426 write 0x28f8600 thru 0x28fae13 (0x2814 bytes) 2024-09-17T23:30:42.329 INFO:teuthology.orchestra.run.smithi139.stdout:4431 trunc from 0x8bfe200 to 0x6308400 2024-09-17T23:30:42.334 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:42.332+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:42.349 INFO:teuthology.orchestra.run.smithi139.stdout:4432 punch from 0x2c40e00 to 0x2c46400, (0x5600 bytes) 2024-09-17T23:30:42.352 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.352 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:42.368 INFO:teuthology.orchestra.run.smithi139.stdout:4433 write 0x9dc0800 thru 0x9dce261 (0xda62 bytes) 2024-09-17T23:30:42.373 INFO:teuthology.orchestra.run.smithi139.stdout:4436 read 0x18c9a00 thru 0x18ced62 (0x5363 bytes) 2024-09-17T23:30:42.374 INFO:teuthology.orchestra.run.smithi139.stdout:4438 write 0x5b9ca00 thru 0x5b9d008 (0x609 bytes) 2024-09-17T23:30:42.375 INFO:teuthology.orchestra.run.smithi139.stdout:4439 read 0x1ab5800 thru 0x1ab6f62 (0x1763 bytes) 2024-09-17T23:30:42.375 INFO:teuthology.orchestra.run.smithi139.stdout:4440 write 0x14fbe00 thru 0x14fdc04 (0x1e05 bytes) 2024-09-17T23:30:42.376 INFO:teuthology.orchestra.run.smithi139.stdout:4441 punch from 0x277b800 to 0x2784600, (0x8e00 bytes) 2024-09-17T23:30:42.376 INFO:teuthology.orchestra.run.smithi139.stdout:4442 write 0xf4a00 thru 0xf5aa3 (0x10a4 bytes) 2024-09-17T23:30:42.377 INFO:teuthology.orchestra.run.smithi139.stdout:4444 punch from 0x77e6a00 to 0x77ebc00, (0x5200 bytes) 2024-09-17T23:30:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:4446 write 0xb604400 thru 0xb611b71 (0xd772 bytes) 2024-09-17T23:30:42.409 INFO:teuthology.orchestra.run.smithi139.stdout:4449 write 0xdae3e00 thru 0xdae56d1 (0x18d2 bytes) 2024-09-17T23:30:42.415 INFO:teuthology.orchestra.run.smithi139.stdout:4450 read 0x1850400 thru 0x1858485 (0x8086 bytes) 2024-09-17T23:30:42.415 INFO:teuthology.orchestra.run.smithi139.stdout:4451 write 0xa63be00 thru 0xa6485a4 (0xc7a5 bytes) 2024-09-17T23:30:42.422 INFO:teuthology.orchestra.run.smithi139.stdout:4453 trunc from 0xdae5800 to 0x8b01600 2024-09-17T23:30:42.442 INFO:teuthology.orchestra.run.smithi139.stdout:4454 punch from 0x41bfe00 to 0x41cf600, (0xf800 bytes) 2024-09-17T23:30:42.445 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:42.443+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:42.448 INFO:teuthology.orchestra.run.smithi139.stdout:4457 read 0x3b79400 thru 0x3b7c26c (0x2e6d bytes) 2024-09-17T23:30:42.449 INFO:teuthology.orchestra.run.smithi139.stdout:4458 punch from 0x1f04a00 to 0x1f0d800, (0x8e00 bytes) 2024-09-17T23:30:42.449 INFO:teuthology.orchestra.run.smithi139.stdout:4459 read 0x598f200 thru 0x5991e60 (0x2c61 bytes) 2024-09-17T23:30:42.450 INFO:teuthology.orchestra.run.smithi139.stdout:4461 read 0x3b9c600 thru 0x3ba124a (0x4c4b bytes) 2024-09-17T23:30:42.452 INFO:teuthology.orchestra.run.smithi139.stdout:4465 punch from 0x5c04600 to 0x5c0fa00, (0xb400 bytes) 2024-09-17T23:30:42.458 INFO:teuthology.orchestra.run.smithi139.stdout:4466 trunc from 0x8b01600 to 0xd20a800 2024-09-17T23:30:42.466 INFO:teuthology.orchestra.run.smithi139.stdout:4468 read 0x23d7200 thru 0x23dfa60 (0x8861 bytes) 2024-09-17T23:30:42.484 INFO:teuthology.orchestra.run.smithi139.stdout:4469 write 0xdf9c600 thru 0xdfaa155 (0xdb56 bytes) 2024-09-17T23:30:42.489 INFO:teuthology.orchestra.run.smithi139.stdout:4470 flatten 2024-09-17T23:30:42.501 INFO:teuthology.orchestra.run.smithi139.stdout:4473 punch from 0x9686a00 to 0x9687400, (0xa00 bytes) 2024-09-17T23:30:42.503 INFO:teuthology.orchestra.run.smithi139.stdout:4474 punch from 0x66dd200 to 0x66e7e00, (0xac00 bytes) 2024-09-17T23:30:42.503 INFO:teuthology.orchestra.run.smithi139.stdout:4475 punch from 0xbc7e800 to 0xbc81400, (0x2c00 bytes) 2024-09-17T23:30:42.503 INFO:teuthology.orchestra.run.smithi139.stdout:4480 read 0xd482a00 thru 0xd49149d (0xea9e bytes) 2024-09-17T23:30:42.503 INFO:teuthology.orchestra.run.smithi139.stdout:4481 read 0x706b000 thru 0x706d09e (0x209f bytes) 2024-09-17T23:30:42.503 INFO:teuthology.orchestra.run.smithi139.stdout:4483 punch from 0xc692400 to 0xc6a0200, (0xde00 bytes) 2024-09-17T23:30:42.503 INFO:teuthology.orchestra.run.smithi139.stdout:4486 punch from 0xa0f6000 to 0xa104200, (0xe200 bytes) 2024-09-17T23:30:42.504 INFO:teuthology.orchestra.run.smithi139.stdout:4487 read 0x4677200 thru 0x468647c (0xf27d bytes) 2024-09-17T23:30:42.504 INFO:teuthology.orchestra.run.smithi139.stdout:4491 read 0x8d1e000 thru 0x8d22824 (0x4825 bytes) 2024-09-17T23:30:42.505 INFO:teuthology.orchestra.run.smithi139.stdout:4495 write 0x2c3da00 thru 0x2c42f15 (0x5516 bytes) 2024-09-17T23:30:42.509 INFO:teuthology.orchestra.run.smithi139.stdout:4497 read 0x3176800 thru 0x317f6b4 (0x8eb5 bytes) 2024-09-17T23:30:42.514 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.514 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.520 INFO:teuthology.orchestra.run.smithi139.stdout:4499 write 0xecd8800 thru 0xece0179 (0x797a bytes) 2024-09-17T23:30:42.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:42.526 INFO:teuthology.orchestra.run.smithi139.stdout:4504 write 0xbc2ec00 thru 0xbc34ad4 (0x5ed5 bytes) 2024-09-17T23:30:42.526 INFO:teuthology.orchestra.run.smithi139.stdout:4508 read 0x8456a00 thru 0x846484b (0xde4c bytes) 2024-09-17T23:30:42.527 INFO:teuthology.orchestra.run.smithi139.stdout:4510 read 0x1fd0c00 thru 0x1fd4a0f (0x3e10 bytes) 2024-09-17T23:30:42.527 INFO:teuthology.orchestra.run.smithi139.stdout:4511 write 0x7b64600 thru 0x7b697bd (0x51be bytes) 2024-09-17T23:30:42.528 INFO:teuthology.orchestra.run.smithi139.stdout:4512 write 0x9f58600 thru 0x9f5b469 (0x2e6a bytes) 2024-09-17T23:30:42.528 INFO:teuthology.orchestra.run.smithi139.stdout:4518 read 0x25e6c00 thru 0x25f58f2 (0xecf3 bytes) 2024-09-17T23:30:42.529 INFO:teuthology.orchestra.run.smithi139.stdout:4519 read 0xdabec00 thru 0xdacd3e3 (0xe7e4 bytes) 2024-09-17T23:30:42.529 INFO:teuthology.orchestra.run.smithi139.stdout:4521 write 0x4925c00 thru 0x492be15 (0x6216 bytes) 2024-09-17T23:30:42.530 INFO:teuthology.orchestra.run.smithi139.stdout:4522 read 0x36f3a00 thru 0x36fc877 (0x8e78 bytes) 2024-09-17T23:30:42.530 INFO:teuthology.orchestra.run.smithi139.stdout:4524 write 0x7435a00 thru 0x7437c3c (0x223d bytes) 2024-09-17T23:30:42.531 INFO:teuthology.orchestra.run.smithi139.stdout:4525 read 0x49f7000 thru 0x4a04322 (0xd323 bytes) 2024-09-17T23:30:42.531 INFO:teuthology.orchestra.run.smithi139.stdout:4530 write 0x107fc00 thru 0x1081b9a (0x1f9b bytes) 2024-09-17T23:30:42.532 INFO:teuthology.orchestra.run.smithi139.stdout:4531 punch from 0xa3a4c00 to 0xa3a6400, (0x1800 bytes) 2024-09-17T23:30:42.550 INFO:teuthology.orchestra.run.smithi139.stdout:4532 trunc from 0xece0200 to 0x6d4400 2024-09-17T23:30:42.566 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:42.563+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:42.598 INFO:teuthology.orchestra.run.smithi139.stdout:4533 write 0x7aef200 thru 0x7af233c (0x313d bytes) 2024-09-17T23:30:42.606 INFO:teuthology.orchestra.run.smithi139.stdout:4538 read 0x45de000 thru 0x45eca1f (0xea20 bytes) 2024-09-17T23:30:42.606 INFO:teuthology.orchestra.run.smithi139.stdout:4539 read 0xa8fa00 thru 0xa9c89a (0xce9b bytes) 2024-09-17T23:30:42.607 INFO:teuthology.orchestra.run.smithi139.stdout:4545 read 0x17d0600 thru 0x17d69e4 (0x63e5 bytes) 2024-09-17T23:30:42.607 INFO:teuthology.orchestra.run.smithi139.stdout:4546 read 0x55c4200 thru 0x55cab01 (0x6902 bytes) 2024-09-17T23:30:42.608 INFO:teuthology.orchestra.run.smithi139.stdout:4547 read 0x3a28800 thru 0x3a2b0ec (0x28ed bytes) 2024-09-17T23:30:42.608 INFO:teuthology.orchestra.run.smithi139.stdout:4550 punch from 0xe8ba00 to 0xe99000, (0xd600 bytes) 2024-09-17T23:30:42.618 INFO:teuthology.orchestra.run.smithi139.stdout:4553 write 0x7d5e400 thru 0x7d6a865 (0xc466 bytes) 2024-09-17T23:30:42.638 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:42.635+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:42.647 INFO:teuthology.orchestra.run.smithi139.stdout:4554 write 0xe924a00 thru 0xe92f5c0 (0xabc1 bytes) 2024-09-17T23:30:42.655 INFO:teuthology.orchestra.run.smithi139.stdout:4555 write 0x7a1e000 thru 0x7a23d3c (0x5d3d bytes) 2024-09-17T23:30:42.655 INFO:teuthology.orchestra.run.smithi139.stdout:4556 read 0xc925600 thru 0xc92c25f (0x6c60 bytes) 2024-09-17T23:30:42.655 INFO:teuthology.orchestra.run.smithi139.stdout:4557 write 0x225da00 thru 0x2261bbe (0x41bf bytes) 2024-09-17T23:30:42.669 INFO:teuthology.orchestra.run.smithi139.stdout:4559 trunc from 0xe92f600 to 0x2ca2200 2024-09-17T23:30:42.677 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.677 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:42.709 INFO:teuthology.orchestra.run.smithi139.stdout:4560 trunc from 0x2ca2200 to 0xd929600 2024-09-17T23:30:42.724 INFO:teuthology.orchestra.run.smithi139.stdout:4562 write 0xc108600 thru 0xc117b86 (0xf587 bytes) 2024-09-17T23:30:42.724 INFO:teuthology.orchestra.run.smithi139.stdout:4563 read 0xd53f200 thru 0xd545d78 (0x6b79 bytes) 2024-09-17T23:30:42.724 INFO:teuthology.orchestra.run.smithi139.stdout:4564 write 0x5ae5a00 thru 0x5ae83af (0x29b0 bytes) 2024-09-17T23:30:42.724 INFO:teuthology.orchestra.run.smithi139.stdout:4565 punch from 0x1918200 to 0x1919e00, (0x1c00 bytes) 2024-09-17T23:30:42.725 INFO:teuthology.orchestra.run.smithi139.stdout:4568 write 0x40b6600 thru 0x40b976c (0x316d bytes) 2024-09-17T23:30:42.727 INFO:teuthology.orchestra.run.smithi139.stdout:4569 read 0xd38a00 thru 0xd46bfc (0xe1fd bytes) 2024-09-17T23:30:42.732 INFO:teuthology.orchestra.run.smithi139.stdout:4571 trunc from 0xd929600 to 0xacaaa00 2024-09-17T23:30:42.738 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:42.735+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:42.748 INFO:teuthology.orchestra.run.smithi139.stdout:4572 punch from 0x6f7e00 to 0x702a00, (0xac00 bytes) 2024-09-17T23:30:42.756 INFO:teuthology.orchestra.run.smithi139.stdout:4574 punch from 0x24dc000 to 0x24e0c00, (0x4c00 bytes) 2024-09-17T23:30:42.756 INFO:teuthology.orchestra.run.smithi139.stdout:4575 write 0x59b4800 thru 0x59bf99d (0xb19e bytes) 2024-09-17T23:30:42.773 INFO:teuthology.orchestra.run.smithi139.stdout:4577 write 0xe2b5800 thru 0xe2ba99a (0x519b bytes) 2024-09-17T23:30:42.779 INFO:teuthology.orchestra.run.smithi139.stdout:4580 read 0x34b0000 thru 0x34b2b58 (0x2b59 bytes) 2024-09-17T23:30:42.779 INFO:teuthology.orchestra.run.smithi139.stdout:4582 write 0x6a3b000 thru 0x6a3b3cf (0x3d0 bytes) 2024-09-17T23:30:42.780 INFO:teuthology.orchestra.run.smithi139.stdout:4583 write 0xa330000 thru 0xa33be67 (0xbe68 bytes) 2024-09-17T23:30:42.780 INFO:teuthology.orchestra.run.smithi139.stdout:4586 punch from 0x463d600 to 0x464a600, (0xd000 bytes) 2024-09-17T23:30:42.780 INFO:teuthology.orchestra.run.smithi139.stdout:4590 read 0x3f2e600 thru 0x3f340db (0x5adc bytes) 2024-09-17T23:30:42.788 INFO:teuthology.orchestra.run.smithi139.stdout:4594 trunc from 0xe2baa00 to 0x7e62000 2024-09-17T23:30:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:42.812 INFO:teuthology.orchestra.run.smithi139.stdout:4596 trunc from 0x7e62000 to 0x47c8800 2024-09-17T23:30:42.839 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:42.836+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:42.844 INFO:teuthology.orchestra.run.smithi139.stdout:4597 write 0x86cbc00 thru 0x86cf189 (0x358a bytes) 2024-09-17T23:30:42.851 INFO:teuthology.orchestra.run.smithi139.stdout:4598 trunc from 0x86cf200 to 0x9e22800 2024-09-17T23:30:42.862 INFO:teuthology.orchestra.run.smithi139.stdout:4601 read 0x6648e00 thru 0x664f821 (0x6a22 bytes) 2024-09-17T23:30:42.870 INFO:teuthology.orchestra.run.smithi139.stdout:4602 punch from 0x2cd5000 to 0x2cdda00, (0x8a00 bytes) 2024-09-17T23:30:42.870 INFO:teuthology.orchestra.run.smithi139.stdout:4603 write 0x834ae00 thru 0x83502f9 (0x54fa bytes) 2024-09-17T23:30:42.870 INFO:teuthology.orchestra.run.smithi139.stdout:4604 clone 30 order 24 su 2097152 sc 9 2024-09-17T23:30:42.918 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:42.918 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:42.923 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:42.924 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:42.924 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T23:30:42.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:30:42.925 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:30:42.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:42.940 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:42.937+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:43.041 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:43.038+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:43.108 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:43.108 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:43.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:43.164 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:43.161+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:43.229 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:43.229 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:43.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:43.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:43.253 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:43.253 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 89, 2024-09-17T23:30:43.253 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:30:43.253 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:30:42.919152+0000", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:28:57.456516+0000", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:30:37.898327+0000", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:30:43.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:30:43.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:43.256 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:43.257 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:43.258 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:43.259 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.260 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:43.261 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.262 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.263 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:43.265 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:43.266 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:43.267 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:43.268 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.2400000095367432, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.2400000095367432, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:43.269 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:30:43.270 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:30:43.271 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "89", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:30:43.272 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 32, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 89, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:30:43.273 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:30:43.274 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.6099998950958252, 2024-09-17T23:30:43.275 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.6099998950958252, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.72906243801116943, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.73000001907348633, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.73000001907348633 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:30:43.276 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 87, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.277 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:30:43.281 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:30:43.282 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 68, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:30:43.287 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:30:43.288 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:43.293 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:43.294 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 87, 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:30:43.296 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:30:43.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:30:43.299 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:30:43.300 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.858154296875, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 87, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.301 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:43.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:30:43.305 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:30:43.306 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.635986328125, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 87, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:30:43.308 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.309 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:30:43.310 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1870185623 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1870185623", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1870185623", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1870185623", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1870185623", 2024-09-17T23:30:43.311 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:43.312 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 55177, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:43.314 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:30:43.315 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:43.316 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:30:43.317 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:30:43.318 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.319 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:43.320 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:43.250+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:43.321 INFO:tasks.thrashosds.thrasher:j is {'epoch': 89, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:30:42.919152+0000', 'last_up_change': '2024-09-17T23:28:57.456516+0000', 'last_in_change': '2024-09-17T23:30:37.898327+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 89, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.609999895095825, 'score_stable': 2.609999895095825, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7290624380111694, 'average_primary_affinity': 0.7300000190734863, 'average_primary_affinity_weighted': 0.7300000190734863}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 0.858154296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 0.635986328125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1870185623}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1870185623}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1870185623}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1870185623}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1870185623}]}, 'public_addr': '172.21.15.37:6809/1870185623', 'cluster_addr': '172.21.15.37:6811/1870185623', 'heartbeat_back_addr': '172.21.15.37:6815/1870185623', 'heartbeat_front_addr': '172.21.15.37:6813/1870185623', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 55177, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.17', 'osds': [5, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:30:43.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:30:43.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:43.344 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T23:30:43.362 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:43.359+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:43.445 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:43.442+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:43.454 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:43.454 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:43.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:43.508 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:43.533 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:43.546 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:43.543+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:43.556 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:43.582 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:43.605 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:43.605 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:43.631 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:43.631 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:43.647 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:43.644+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.610509798999999, 2024-09-17T23:30:43.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.610336859, 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:43.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:43.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:43.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.610296988999998, 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:43.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:43.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:43.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 63.315792762000001, 2024-09-17T23:30:43.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:43.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:43.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 58.068091213000002, 2024-09-17T23:30:43.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:43.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.058158947000003, 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:43.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:43.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:43.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:43.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:43.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.044931898000002, 2024-09-17T23:30:43.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:43.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:43.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:43.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:43.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.504402742000003, 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:43.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.504381352000003, 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.504364223000003, 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.504346941000001, 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.504329912000003, 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.504297037999997, 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.503709332, 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.481917191000001, 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.481380002999998, 2024-09-17T23:30:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.480893887000001, 2024-09-17T23:30:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.408751285000001, 2024-09-17T23:30:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.408729474000001, 2024-09-17T23:30:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.399286604, 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:43.786 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:43.787 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:43.784+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:43.788 INFO:teuthology.orchestra.run.smithi139.stdout:4605 write 0x4219a00 thru 0x421db04 (0x4105 bytes) 2024-09-17T23:30:43.788 INFO:teuthology.orchestra.run.smithi139.stdout:4606 read 0x350de00 thru 0x3513e1e (0x601f bytes) 2024-09-17T23:30:43.788 INFO:teuthology.orchestra.run.smithi139.stdout:4608 write 0x757e400 thru 0x75867ee (0x83ef bytes) 2024-09-17T23:30:43.788 INFO:teuthology.orchestra.run.smithi139.stdout:4610 write 0xa0ca600 thru 0xa0d6ddd (0xc7de bytes) 2024-09-17T23:30:43.788 INFO:teuthology.orchestra.run.smithi139.stdout:4615 write 0xc3b7c00 thru 0xc3c3ebe (0xc2bf bytes) 2024-09-17T23:30:43.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:30:43.791 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:30:43.792 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-17T23:30:42.923787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1023827192,"num_objects":1158,"num_object_clones":343,"num_object_copies":2316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1158,"num_whiteouts":387,"num_read":213185,"num_read_kb":819687,"num_write":11816,"num_write_kb":323725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1501,"num_bytes_recovered":53129311,"num_keys_recovered":316,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10465,"ondisk_log_size":10465,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":210284,"kb_used_data":25708,"kb_used_omap":6,"kb_used_meta":181561,"kb_avail":377277076,"statfs":{"total":386547056640,"available":386331725824,"internally_reserved":0,"allocated":26324992,"data_stored":165650891,"data_compressed":7682896,"data_compressed_allocated":16908288,"data_compressed_original":161034752,"omap_allocated":6376,"internal_metadata":185919256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015004"},"pg_stats":[{"pgid":"3.1f","version":"81'319","reported_seq":4792,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.770383+0000","last_change":"2024-09-17T23:30:37.907110+0000","last_active":"2024-09-17T23:30:37.907110+0000","last_peered":"2024-09-17T23:30:37.770383+0000","last_clean":"2024-09-17T23:30:37.770383+0000","last_became_active":"2024-09-17T23:29:15.605286+0000","last_became_peered":"2024-09-17T23:29:15.605286+0000","last_unstale":"2024-09-17T23:30:37.770383+0000","last_undegraded":"2024-09-17T23:30:37.770383+0000","last_fullsized":"2024-09-17T23:30:37.770383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055557299999999996,"stat_sum":{"num_bytes":32570420,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":5091,"num_read_kb":28062,"num_write":354,"num_write_kb":5313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"83'401","reported_seq":6828,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.768753+0000","last_change":"2024-09-17T23:30:37.905106+0000","last_active":"2024-09-17T23:30:37.905106+0000","last_peered":"2024-09-17T23:30:37.768753+0000","last_clean":"2024-09-17T23:30:37.768753+0000","last_became_active":"2024-09-17T23:30:21.854231+0000","last_became_peered":"2024-09-17T23:30:21.854231+0000","last_unstale":"2024-09-17T23:30:37.768753+0000","last_undegraded":"2024-09-17T23:30:37.768753+0000","last_fullsized":"2024-09-17T23:30:37.768753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055521100000000001,"stat_sum":{"num_bytes":82797394,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":8074,"num_read_kb":38578,"num_write":416,"num_write_kb":9413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":8673105,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"83'207","reported_seq":3154,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.165257+0000","last_change":"2024-09-17T23:30:37.906686+0000","last_active":"2024-09-17T23:30:37.906686+0000","last_peered":"2024-09-17T23:30:37.165257+0000","last_clean":"2024-09-17T23:30:37.165257+0000","last_became_active":"2024-09-17T23:30:21.854727+0000","last_became_peered":"2024-09-17T23:30:21.854727+0000","last_unstale":"2024-09-17T23:30:37.165257+0000","last_undegraded":"2024-09-17T23:30:37.165257+0000","last_fullsized":"2024-09-17T23:30:37.165257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000476274,"stat_sum":{"num_bytes":18891814,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":3006,"num_read_kb":24279,"num_write":211,"num_write_kb":9547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":1889280,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"83'154","reported_seq":6603,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.900289+0000","last_change":"2024-09-17T23:30:37.906010+0000","last_active":"2024-09-17T23:30:37.906010+0000","last_peered":"2024-09-17T23:30:37.900289+0000","last_clean":"2024-09-17T23:30:37.900289+0000","last_became_active":"2024-09-17T23:29:39.959404+0000","last_became_peered":"2024-09-17T23:29:39.959404+0000","last_unstale":"2024-09-17T23:30:37.900289+0000","last_undegraded":"2024-09-17T23:30:37.900289+0000","last_fullsized":"2024-09-17T23:30:37.900289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001263599,"stat_sum":{"num_bytes":21713424,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":10,"num_read":6335,"num_read_kb":20601,"num_write":153,"num_write_kb":9947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"79'257","reported_seq":15967,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904111+0000","last_change":"2024-09-17T23:29:16.844175+0000","last_active":"2024-09-17T23:30:37.904111+0000","last_peered":"2024-09-17T23:30:37.904111+0000","last_clean":"2024-09-17T23:30:37.904111+0000","last_became_active":"2024-09-17T23:29:15.601606+0000","last_became_peered":"2024-09-17T23:29:15.601606+0000","last_unstale":"2024-09-17T23:30:37.904111+0000","last_undegraded":"2024-09-17T23:30:37.904111+0000","last_fullsized":"2024-09-17T23:30:37.904111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:19:40.652687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046809400000000001,"stat_sum":{"num_bytes":67564347,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":15983,"num_read_kb":41116,"num_write":302,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"81'600","reported_seq":2835,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.904399+0000","last_change":"2024-09-17T23:30:37.908895+0000","last_active":"2024-09-17T23:30:37.908895+0000","last_peered":"2024-09-17T23:30:37.904399+0000","last_clean":"2024-09-17T23:30:37.904399+0000","last_became_active":"2024-09-17T23:29:59.041866+0000","last_became_peered":"2024-09-17T23:29:59.041866+0000","last_unstale":"2024-09-17T23:30:37.904399+0000","last_undegraded":"2024-09-17T23:30:37.904399+0000","last_fullsized":"2024-09-17T23:30:37.904399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001041527,"stat_sum":{"num_bytes":33840829,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":6707,"num_read_kb":16107,"num_write":1016,"num_write_kb":2718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":179747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"83'250","reported_seq":2908,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.174465+0000","last_change":"2024-09-17T23:30:37.904743+0000","last_active":"2024-09-17T23:30:37.904743+0000","last_peered":"2024-09-17T23:30:37.174465+0000","last_clean":"2024-09-17T23:30:37.174465+0000","last_became_active":"2024-09-17T23:30:28.866986+0000","last_became_peered":"2024-09-17T23:30:28.866986+0000","last_unstale":"2024-09-17T23:30:37.174465+0000","last_undegraded":"2024-09-17T23:30:37.174465+0000","last_fullsized":"2024-09-17T23:30:37.174465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001306843,"stat_sum":{"num_bytes":26913876,"num_objects":37,"num_object_clones":12,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":3349,"num_read_kb":6994,"num_write":214,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":971311,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"81'295","reported_seq":2997,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904017+0000","last_change":"2024-09-17T23:29:16.844092+0000","last_active":"2024-09-17T23:30:37.904017+0000","last_peered":"2024-09-17T23:30:37.904017+0000","last_clean":"2024-09-17T23:30:37.904017+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:30:37.904017+0000","last_undegraded":"2024-09-17T23:30:37.904017+0000","last_fullsized":"2024-09-17T23:30:37.904017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:25:35.204812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045869699999999999,"stat_sum":{"num_bytes":18727476,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":3489,"num_read_kb":17210,"num_write":328,"num_write_kb":18054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"82'441","reported_seq":7020,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906626+0000","last_change":"2024-09-17T23:30:09.095371+0000","last_active":"2024-09-17T23:30:37.906626+0000","last_peered":"2024-09-17T23:30:37.906626+0000","last_clean":"2024-09-17T23:30:37.906626+0000","last_became_active":"2024-09-17T23:30:09.094524+0000","last_became_peered":"2024-09-17T23:30:09.094524+0000","last_unstale":"2024-09-17T23:30:37.906626+0000","last_undegraded":"2024-09-17T23:30:37.906626+0000","last_fullsized":"2024-09-17T23:30:37.906626+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:22.404098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050274800000000002,"stat_sum":{"num_bytes":37074050,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":8672,"num_read_kb":19545,"num_write":402,"num_write_kb":8265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2316890,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"65'152","reported_seq":2276,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904381+0000","last_change":"2024-09-17T23:30:21.856627+0000","last_active":"2024-09-17T23:30:37.904381+0000","last_peered":"2024-09-17T23:30:37.904381+0000","last_clean":"2024-09-17T23:30:37.904381+0000","last_became_active":"2024-09-17T23:30:21.855956+0000","last_became_peered":"2024-09-17T23:30:21.855956+0000","last_unstale":"2024-09-17T23:30:37.904381+0000","last_undegraded":"2024-09-17T23:30:37.904381+0000","last_fullsized":"2024-09-17T23:30:37.904381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:22.969124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099808899999999996,"stat_sum":{"num_bytes":15285808,"num_objects":29,"num_object_clones":10,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":2559,"num_read_kb":14731,"num_write":149,"num_write_kb":4044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":2490976,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"79'401","reported_seq":10851,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906924+0000","last_change":"2024-09-17T23:29:16.851584+0000","last_active":"2024-09-17T23:30:37.906924+0000","last_peered":"2024-09-17T23:30:37.906924+0000","last_clean":"2024-09-17T23:30:37.906924+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:30:37.906924+0000","last_undegraded":"2024-09-17T23:30:37.906924+0000","last_fullsized":"2024-09-17T23:30:37.906924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:11.172938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073596200000000005,"stat_sum":{"num_bytes":32097361,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":12072,"num_read_kb":22048,"num_write":476,"num_write_kb":3508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"79'435","reported_seq":7376,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905290+0000","last_change":"2024-09-17T23:30:28.867237+0000","last_active":"2024-09-17T23:30:37.905290+0000","last_peered":"2024-09-17T23:30:37.905290+0000","last_clean":"2024-09-17T23:30:37.905290+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:30:37.905290+0000","last_undegraded":"2024-09-17T23:30:37.905290+0000","last_fullsized":"2024-09-17T23:30:37.905290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:22:22.623459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013571150000000001,"stat_sum":{"num_bytes":21155425,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":8257,"num_read_kb":32091,"num_write":379,"num_write_kb":17994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906283+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:30:37.906283+0000","last_peered":"2024-09-17T23:30:37.906283+0000","last_clean":"2024-09-17T23:30:37.906283+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:30:37.906283+0000","last_undegraded":"2024-09-17T23:30:37.906283+0000","last_fullsized":"2024-09-17T23:30:37.906283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"79'467","reported_seq":1542,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.908177+0000","last_change":"2024-09-17T23:29:16.856135+0000","last_active":"2024-09-17T23:30:37.908177+0000","last_peered":"2024-09-17T23:30:37.908177+0000","last_clean":"2024-09-17T23:30:37.908177+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:30:37.908177+0000","last_undegraded":"2024-09-17T23:30:37.908177+0000","last_fullsized":"2024-09-17T23:30:37.908177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:31.035309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001532662,"stat_sum":{"num_bytes":13427287,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2845,"num_read_kb":8702,"num_write":754,"num_write_kb":6043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.908249+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:30:37.908249+0000","last_peered":"2024-09-17T23:30:37.908249+0000","last_clean":"2024-09-17T23:30:37.908249+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:30:37.908249+0000","last_undegraded":"2024-09-17T23:30:37.908249+0000","last_fullsized":"2024-09-17T23:30:37.908249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"79'214","reported_seq":8508,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.908418+0000","last_change":"2024-09-17T23:29:16.855828+0000","last_active":"2024-09-17T23:30:37.908418+0000","last_peered":"2024-09-17T23:30:37.908418+0000","last_clean":"2024-09-17T23:30:37.908418+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:30:37.908418+0000","last_undegraded":"2024-09-17T23:30:37.908418+0000","last_fullsized":"2024-09-17T23:30:37.908418+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:09.902689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012515670000000001,"stat_sum":{"num_bytes":19786283,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":9208,"num_read_kb":29009,"num_write":164,"num_write_kb":20180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:36.913552+0000","last_change":"2024-09-17T23:30:37.908448+0000","last_active":"2024-09-17T23:30:37.908448+0000","last_peered":"2024-09-17T23:30:36.913552+0000","last_clean":"2024-09-17T23:30:36.913552+0000","last_became_active":"2024-09-17T23:29:39.958620+0000","last_became_peered":"2024-09-17T23:29:39.958620+0000","last_unstale":"2024-09-17T23:30:36.913552+0000","last_undegraded":"2024-09-17T23:30:36.913552+0000","last_fullsized":"2024-09-17T23:30:36.913552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"68'250","reported_seq":5837,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.908114+0000","last_change":"2024-09-17T23:30:21.856676+0000","last_active":"2024-09-17T23:30:37.908114+0000","last_peered":"2024-09-17T23:30:37.908114+0000","last_clean":"2024-09-17T23:30:37.908114+0000","last_became_active":"2024-09-17T23:30:21.855597+0000","last_became_peered":"2024-09-17T23:30:21.855597+0000","last_unstale":"2024-09-17T23:30:37.908114+0000","last_undegraded":"2024-09-17T23:30:37.908114+0000","last_fullsized":"2024-09-17T23:30:37.908114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:41:28.089336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010554410000000001,"stat_sum":{"num_bytes":18939983,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":3725,"num_read_kb":13923,"num_write":307,"num_write_kb":4111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":5497404,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":155,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906281+0000","last_change":"2024-09-17T23:30:25.854649+0000","last_active":"2024-09-17T23:30:37.906281+0000","last_peered":"2024-09-17T23:30:37.906281+0000","last_clean":"2024-09-17T23:30:37.906281+0000","last_became_active":"2024-09-17T23:30:25.854287+0000","last_became_peered":"2024-09-17T23:30:25.854287+0000","last_unstale":"2024-09-17T23:30:37.906281+0000","last_undegraded":"2024-09-17T23:30:37.906281+0000","last_fullsized":"2024-09-17T23:30:37.906281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:10:16.445314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"81'172","reported_seq":9131,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904255+0000","last_change":"2024-09-17T23:30:21.852782+0000","last_active":"2024-09-17T23:30:37.904255+0000","last_peered":"2024-09-17T23:30:37.904255+0000","last_clean":"2024-09-17T23:30:37.904255+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:30:37.904255+0000","last_undegraded":"2024-09-17T23:30:37.904255+0000","last_fullsized":"2024-09-17T23:30:37.904255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:43.942352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23530074,"num_objects":44,"num_object_clones":14,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":8755,"num_read_kb":18761,"num_write":171,"num_write_kb":3764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"83'205","reported_seq":3935,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905595+0000","last_change":"2024-09-17T23:30:21.855453+0000","last_active":"2024-09-17T23:30:37.905595+0000","last_peered":"2024-09-17T23:30:37.905595+0000","last_clean":"2024-09-17T23:30:37.905595+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:30:37.905595+0000","last_undegraded":"2024-09-17T23:30:37.905595+0000","last_fullsized":"2024-09-17T23:30:37.905595+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:21:03.908084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000582823,"stat_sum":{"num_bytes":29375012,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":3947,"num_read_kb":14706,"num_write":188,"num_write_kb":4008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":161,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:36.901167+0000","last_change":"2024-09-17T23:30:37.906267+0000","last_active":"2024-09-17T23:30:37.906267+0000","last_peered":"2024-09-17T23:30:36.901167+0000","last_clean":"2024-09-17T23:30:36.901167+0000","last_became_active":"2024-09-17T23:30:25.856761+0000","last_became_peered":"2024-09-17T23:30:25.856761+0000","last_unstale":"2024-09-17T23:30:36.901167+0000","last_undegraded":"2024-09-17T23:30:36.901167+0000","last_fullsized":"2024-09-17T23:30:36.901167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"81'403","reported_seq":5857,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.772636+0000","last_change":"2024-09-17T23:30:37.906651+0000","last_active":"2024-09-17T23:30:37.906651+0000","last_peered":"2024-09-17T23:30:37.772636+0000","last_clean":"2024-09-17T23:30:37.772636+0000","last_became_active":"2024-09-17T23:30:28.866471+0000","last_became_peered":"2024-09-17T23:30:28.866471+0000","last_unstale":"2024-09-17T23:30:37.772636+0000","last_undegraded":"2024-09-17T23:30:37.772636+0000","last_fullsized":"2024-09-17T23:30:37.772636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363521,"stat_sum":{"num_bytes":15584385,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":6891,"num_read_kb":19121,"num_write":519,"num_write_kb":1950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":841008,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904791+0000","last_change":"2024-09-17T23:29:09.377458+0000","last_active":"2024-09-17T23:30:37.904791+0000","last_peered":"2024-09-17T23:30:37.904791+0000","last_clean":"2024-09-17T23:30:37.904791+0000","last_became_active":"2024-09-17T23:29:05.495411+0000","last_became_peered":"2024-09-17T23:29:05.495411+0000","last_unstale":"2024-09-17T23:30:37.904791+0000","last_undegraded":"2024-09-17T23:30:37.904791+0000","last_fullsized":"2024-09-17T23:30:37.904791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:01:59.106784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"82'521","reported_seq":6027,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904033+0000","last_change":"2024-09-17T23:30:28.866647+0000","last_active":"2024-09-17T23:30:37.904033+0000","last_peered":"2024-09-17T23:30:37.904033+0000","last_clean":"2024-09-17T23:30:37.904033+0000","last_became_active":"2024-09-17T23:30:28.864648+0000","last_became_peered":"2024-09-17T23:30:28.864648+0000","last_unstale":"2024-09-17T23:30:37.904033+0000","last_undegraded":"2024-09-17T23:30:37.904033+0000","last_fullsized":"2024-09-17T23:30:37.904033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:37.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00105678,"stat_sum":{"num_bytes":33188027,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":18,"num_read":8433,"num_read_kb":10517,"num_write":337,"num_write_kb":2782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":2400118,"num_keys_recovered":94,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906123+0000","last_change":"2024-09-17T23:30:27.859228+0000","last_active":"2024-09-17T23:30:37.906123+0000","last_peered":"2024-09-17T23:30:37.906123+0000","last_clean":"2024-09-17T23:30:37.906123+0000","last_became_active":"2024-09-17T23:30:27.858922+0000","last_became_peered":"2024-09-17T23:30:27.858922+0000","last_unstale":"2024-09-17T23:30:37.906123+0000","last_undegraded":"2024-09-17T23:30:37.906123+0000","last_fullsized":"2024-09-17T23:30:37.906123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:24.763276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"83'446","reported_seq":4479,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.904183+0000","last_change":"2024-09-17T23:30:21.856494+0000","last_active":"2024-09-17T23:30:37.904183+0000","last_peered":"2024-09-17T23:30:37.904183+0000","last_clean":"2024-09-17T23:30:37.904183+0000","last_became_active":"2024-09-17T23:30:21.855686+0000","last_became_peered":"2024-09-17T23:30:21.855686+0000","last_unstale":"2024-09-17T23:30:37.904183+0000","last_undegraded":"2024-09-17T23:30:37.904183+0000","last_fullsized":"2024-09-17T23:30:37.904183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:24.259530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047998399999999998,"stat_sum":{"num_bytes":32934025,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":5404,"num_read_kb":39932,"num_write":531,"num_write_kb":50362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":3843697,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":47,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:36.913461+0000","last_change":"2024-09-17T23:30:37.908483+0000","last_active":"2024-09-17T23:30:37.908483+0000","last_peered":"2024-09-17T23:30:36.913461+0000","last_clean":"2024-09-17T23:30:36.913461+0000","last_became_active":"2024-09-17T23:30:19.157650+0000","last_became_peered":"2024-09-17T23:30:19.157650+0000","last_unstale":"2024-09-17T23:30:36.913461+0000","last_undegraded":"2024-09-17T23:30:36.913461+0000","last_fullsized":"2024-09-17T23:30:36.913461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":255,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:36.900101+0000","last_change":"2024-09-17T23:30:37.906690+0000","last_active":"2024-09-17T23:30:37.906690+0000","last_peered":"2024-09-17T23:30:36.900101+0000","last_clean":"2024-09-17T23:30:36.900101+0000","last_became_active":"2024-09-17T23:29:39.959373+0000","last_became_peered":"2024-09-17T23:29:39.959373+0000","last_unstale":"2024-09-17T23:30:36.900101+0000","last_undegraded":"2024-09-17T23:30:36.900101+0000","last_fullsized":"2024-09-17T23:30:36.900101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"82'280","reported_seq":11425,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.908028+0000","last_change":"2024-09-17T23:30:21.856806+0000","last_active":"2024-09-17T23:30:37.908028+0000","last_peered":"2024-09-17T23:30:37.908028+0000","last_clean":"2024-09-17T23:30:37.908028+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:30:37.908028+0000","last_undegraded":"2024-09-17T23:30:37.908028+0000","last_fullsized":"2024-09-17T23:30:37.908028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:45:52.425972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001536372,"stat_sum":{"num_bytes":42443021,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":11967,"num_read_kb":59175,"num_write":253,"num_write_kb":5035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"83'386","reported_seq":4669,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905038+0000","last_change":"2024-09-17T23:29:16.850836+0000","last_active":"2024-09-17T23:30:37.905038+0000","last_peered":"2024-09-17T23:30:37.905038+0000","last_clean":"2024-09-17T23:30:37.905038+0000","last_became_active":"2024-09-17T23:29:15.602469+0000","last_became_peered":"2024-09-17T23:29:15.602469+0000","last_unstale":"2024-09-17T23:30:37.905038+0000","last_undegraded":"2024-09-17T23:30:37.905038+0000","last_fullsized":"2024-09-17T23:30:37.905038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:40:50.752804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000483418,"stat_sum":{"num_bytes":14314604,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":4098,"num_read_kb":22764,"num_write":491,"num_write_kb":19775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"83'432","reported_seq":8589,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905638+0000","last_change":"2024-09-17T23:30:30.876410+0000","last_active":"2024-09-17T23:30:37.905638+0000","last_peered":"2024-09-17T23:30:37.905638+0000","last_clean":"2024-09-17T23:30:37.905638+0000","last_became_active":"2024-09-17T23:30:30.874650+0000","last_became_peered":"2024-09-17T23:30:30.874650+0000","last_unstale":"2024-09-17T23:30:37.905638+0000","last_undegraded":"2024-09-17T23:30:37.905638+0000","last_fullsized":"2024-09-17T23:30:37.905638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:03:26.173670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060532200000000004,"stat_sum":{"num_bytes":20512573,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":13,"num_read":10108,"num_read_kb":45148,"num_write":457,"num_write_kb":4891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":2932541,"num_keys_recovered":25,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"83'514","reported_seq":5641,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905823+0000","last_change":"2024-09-17T23:29:16.844405+0000","last_active":"2024-09-17T23:30:37.905823+0000","last_peered":"2024-09-17T23:30:37.905823+0000","last_clean":"2024-09-17T23:30:37.905823+0000","last_became_active":"2024-09-17T23:29:15.601213+0000","last_became_peered":"2024-09-17T23:29:15.601213+0000","last_unstale":"2024-09-17T23:30:37.905823+0000","last_undegraded":"2024-09-17T23:30:37.905823+0000","last_fullsized":"2024-09-17T23:30:37.905823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:35.786371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049268800000000002,"stat_sum":{"num_bytes":37653062,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":7941,"num_read_kb":32217,"num_write":629,"num_write_kb":7454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"81'316","reported_seq":4450,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905392+0000","last_change":"2024-09-17T23:29:16.851583+0000","last_active":"2024-09-17T23:30:37.905392+0000","last_peered":"2024-09-17T23:30:37.905392+0000","last_clean":"2024-09-17T23:30:37.905392+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:30:37.905392+0000","last_undegraded":"2024-09-17T23:30:37.905392+0000","last_fullsized":"2024-09-17T23:30:37.905392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:21:19.884738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080187900000000002,"stat_sum":{"num_bytes":21493439,"num_objects":45,"num_object_clones":16,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":5386,"num_read_kb":33545,"num_write":333,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"81'160","reported_seq":7299,"reported_epoch":84,"state":"unknown","last_fresh":"2024-09-17T23:30:37.907415+0000","last_change":"2024-09-17T23:30:38.397698+0000","last_active":"2024-09-17T23:30:37.907415+0000","last_peered":"2024-09-17T23:30:34.545306+0000","last_clean":"2024-09-17T23:30:34.545306+0000","last_became_active":"2024-09-17T23:29:15.599701+0000","last_became_peered":"2024-09-17T23:29:15.599701+0000","last_unstale":"2024-09-17T23:30:37.907415+0000","last_undegraded":"2024-09-17T23:30:37.907415+0000","last_fullsized":"2024-09-17T23:30:37.907415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001095122,"stat_sum":{"num_bytes":30469700,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":7098,"num_read_kb":23693,"num_write":171,"num_write_kb":9140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"81'480","reported_seq":7495,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906165+0000","last_change":"2024-09-17T23:30:21.855471+0000","last_active":"2024-09-17T23:30:37.906165+0000","last_peered":"2024-09-17T23:30:37.906165+0000","last_clean":"2024-09-17T23:30:37.906165+0000","last_became_active":"2024-09-17T23:30:21.854475+0000","last_became_peered":"2024-09-17T23:30:21.854475+0000","last_unstale":"2024-09-17T23:30:37.906165+0000","last_undegraded":"2024-09-17T23:30:37.906165+0000","last_fullsized":"2024-09-17T23:30:37.906165+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:09:38.095174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093123200000000004,"stat_sum":{"num_bytes":19644124,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":8007,"num_read_kb":19017,"num_write":521,"num_write_kb":3664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1763020,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"81'129","reported_seq":2714,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.906155+0000","last_change":"2024-09-17T23:30:30.876352+0000","last_active":"2024-09-17T23:30:37.906155+0000","last_peered":"2024-09-17T23:30:37.906155+0000","last_clean":"2024-09-17T23:30:37.906155+0000","last_became_active":"2024-09-17T23:30:30.875596+0000","last_became_peered":"2024-09-17T23:30:30.875596+0000","last_unstale":"2024-09-17T23:30:37.906155+0000","last_undegraded":"2024-09-17T23:30:37.906155+0000","last_fullsized":"2024-09-17T23:30:37.906155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:37:55.639695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078560200000000005,"stat_sum":{"num_bytes":9040964,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":2434,"num_read_kb":17493,"num_write":155,"num_write_kb":3561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1477700,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"81'317","reported_seq":2505,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:36.900013+0000","last_change":"2024-09-17T23:30:37.905688+0000","last_active":"2024-09-17T23:30:37.905688+0000","last_peered":"2024-09-17T23:30:36.900013+0000","last_clean":"2024-09-17T23:30:36.900013+0000","last_became_active":"2024-09-17T23:29:59.037097+0000","last_became_peered":"2024-09-17T23:29:59.037097+0000","last_unstale":"2024-09-17T23:30:36.900013+0000","last_undegraded":"2024-09-17T23:30:36.900013+0000","last_fullsized":"2024-09-17T23:30:36.900013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056375800000000003,"stat_sum":{"num_bytes":23871113,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":3776,"num_read_kb":18340,"num_write":356,"num_write_kb":18900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":573478,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"83'181","reported_seq":4121,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:37.898517+0000","last_change":"2024-09-17T23:30:37.904566+0000","last_active":"2024-09-17T23:30:37.904566+0000","last_peered":"2024-09-17T23:30:37.898517+0000","last_clean":"2024-09-17T23:30:37.898517+0000","last_became_active":"2024-09-17T23:30:21.853692+0000","last_became_peered":"2024-09-17T23:30:21.853692+0000","last_unstale":"2024-09-17T23:30:37.898517+0000","last_undegraded":"2024-09-17T23:30:37.898517+0000","last_fullsized":"2024-09-17T23:30:37.898517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043017799999999997,"stat_sum":{"num_bytes":88071680,"num_objects":38,"num_object_clones":13,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":4670,"num_read_kb":31014,"num_write":169,"num_write_kb":4520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3621888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"83'173","reported_seq":3015,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T23:30:37.905528+0000","last_change":"2024-09-17T23:29:59.135036+0000","last_active":"2024-09-17T23:30:37.905528+0000","last_peered":"2024-09-17T23:30:37.905528+0000","last_clean":"2024-09-17T23:30:37.905528+0000","last_became_active":"2024-09-17T23:29:59.042434+0000","last_became_peered":"2024-09-17T23:29:59.042434+0000","last_unstale":"2024-09-17T23:30:37.905528+0000","last_undegraded":"2024-09-17T23:30:37.905528+0000","last_fullsized":"2024-09-17T23:30:37.905528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:40:56.826629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034516700000000001,"stat_sum":{"num_bytes":70821888,"num_objects":31,"num_object_clones":12,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":14,"num_read":2667,"num_read_kb":16449,"num_write":187,"num_write_kb":6821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":3587584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"81'387","reported_seq":11567,"reported_epoch":83,"state":"peering","last_fresh":"2024-09-17T23:30:36.900765+0000","last_change":"2024-09-17T23:30:37.906045+0000","last_active":"2024-09-17T23:30:37.906045+0000","last_peered":"2024-09-17T23:30:36.900765+0000","last_clean":"2024-09-17T23:30:36.900765+0000","last_became_active":"2024-09-17T23:29:39.959472+0000","last_became_peered":"2024-09-17T23:29:39.959472+0000","last_unstale":"2024-09-17T23:30:36.900765+0000","last_undegraded":"2024-09-17T23:30:36.900765+0000","last_fullsized":"2024-09-17T23:30:36.900765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001102308,"stat_sum":{"num_bytes":49503337,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":12,"num_read":12115,"num_read_kb":64699,"num_write":562,"num_write_kb":29267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1023236805,"num_objects":1154,"num_object_clones":343,"num_object_copies":2308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1154,"num_whiteouts":387,"num_read":213069,"num_read_kb":819587,"num_write":11655,"num_write_kb":321307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1497,"num_bytes_recovered":53129292,"num_keys_recovered":316,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":31141888,"data_stored":214293782,"data_compressed":11245753,"data_compressed_allocated":23117824,"data_compressed_original":209863680,"omap_allocated":0,"internal_metadata":0},"log_size":10345,"ondisk_log_size":10345,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9884,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640279,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2519040,"data_stored":11758157,"data_compressed":515291,"data_compressed_allocated":1150976,"data_compressed_original":11138560,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640279,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48404,"kb_used_data":4736,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94323436,"statfs":{"total":96636764160,"available":96587198464,"internally_reserved":0,"allocated":4849664,"data_stored":27584772,"data_compressed":1380820,"data_compressed_allocated":2842624,"data_compressed_original":26658304,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640279,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58876,"kb_used_data":7840,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94312964,"statfs":{"total":96636764160,"available":96576475136,"internally_reserved":0,"allocated":8028160,"data_stored":49359507,"data_compressed":2489969,"data_compressed_allocated":5169152,"data_compressed_original":47928320,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640279,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640280,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640279,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65196,"kb_used_data":10672,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94306644,"statfs":{"total":96636764160,"available":96570003456,"internally_reserved":0,"allocated":10928128,"data_stored":76948455,"data_compressed":3296816,"data_compressed_allocated":7745536,"data_compressed_original":75309568,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9003008,"data_stored":61471259,"data_compressed":3279812,"data_compressed_allocated":6799360,"data_compressed_original":60233216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4431872,"data_stored":31716996,"data_compressed":1598684,"data_compressed_allocated":3297280,"data_compressed_original":31071232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7045120,"data_stored":48960515,"data_compressed":2489969,"data_compressed_allocated":5169152,"data_compressed_original":47928320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3866624,"data_stored":27185780,"data_compressed":1380820,"data_compressed_allocated":2842624,"data_compressed_original":26658304,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1490944,"data_stored":10774132,"data_compressed":510349,"data_compressed_allocated":1114112,"data_compressed_original":10565120,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T23:30:43.795 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [2, 1] 2024-09-17T23:30:43.795 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '2', '1'] 2024-09-17T23:30:43.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 2 1 2024-09-17T23:30:43.797 INFO:teuthology.orchestra.run.smithi139.stdout:4616 write 0x5c92400 thru 0x5c9a53a (0x813b bytes) 2024-09-17T23:30:43.797 INFO:teuthology.orchestra.run.smithi139.stdout:4617 write 0x1097000 thru 0x10a5827 (0xe828 bytes) 2024-09-17T23:30:43.797 INFO:teuthology.orchestra.run.smithi139.stdout:4618 write 0x1014e00 thru 0x101d913 (0x8b14 bytes) 2024-09-17T23:30:43.797 INFO:teuthology.orchestra.run.smithi139.stdout:4620 write 0x9066000 thru 0x9066760 (0x761 bytes) 2024-09-17T23:30:43.797 INFO:teuthology.orchestra.run.smithi139.stdout:4621 read 0xcece00 thru 0xcf467e (0x787f bytes) 2024-09-17T23:30:43.797 INFO:teuthology.orchestra.run.smithi139.stdout:4625 write 0xa2c200 thru 0xa2d4a1 (0x12a2 bytes) 2024-09-17T23:30:43.798 INFO:teuthology.orchestra.run.smithi139.stdout:4631 punch from 0x34e4c00 to 0x34e9600, (0x4a00 bytes) 2024-09-17T23:30:43.799 INFO:teuthology.orchestra.run.smithi139.stdout:4632 read 0xbbf2c00 thru 0xbbf7384 (0x4785 bytes) 2024-09-17T23:30:43.799 INFO:teuthology.orchestra.run.smithi139.stdout:4635 write 0x5760600 thru 0x5761f29 (0x192a bytes) 2024-09-17T23:30:43.803 INFO:teuthology.orchestra.run.smithi139.stdout:4637 read 0x757ba00 thru 0x7585835 (0x9e36 bytes) 2024-09-17T23:30:43.804 INFO:teuthology.orchestra.run.smithi139.stdout:4638 flatten 2024-09-17T23:30:43.842 INFO:teuthology.orchestra.run.smithi139.stdout:4639 punch from 0x1e0fa00 to 0x1e1b600, (0xbc00 bytes) 2024-09-17T23:30:43.842 INFO:teuthology.orchestra.run.smithi139.stdout:4640 punch from 0x9a32e00 to 0x9a38a00, (0x5c00 bytes) 2024-09-17T23:30:43.843 INFO:teuthology.orchestra.run.smithi139.stdout:4642 read 0x745ea00 thru 0x746230a (0x390b bytes) 2024-09-17T23:30:43.843 INFO:teuthology.orchestra.run.smithi139.stdout:4646 write 0x715f600 thru 0x7165f2b (0x692c bytes) 2024-09-17T23:30:43.857 INFO:teuthology.orchestra.run.smithi139.stdout:4647 write 0xdc53400 thru 0xdc58527 (0x5128 bytes) 2024-09-17T23:30:43.858 INFO:teuthology.orchestra.run.smithi139.stdout:4648 write 0x4f9bc00 thru 0x4fa23e7 (0x67e8 bytes) 2024-09-17T23:30:43.864 INFO:teuthology.orchestra.run.smithi139.stdout:4649 trunc from 0xdc58600 to 0x8b8b400 2024-09-17T23:30:43.888 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:43.885+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:43.892 INFO:teuthology.orchestra.run.smithi139.stdout:4650 trunc from 0x8b8b400 to 0x406ba00 2024-09-17T23:30:43.916 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:43.916 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:43.916 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:43.916 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:43.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:43.929 INFO:teuthology.orchestra.run.smithi139.stdout:4651 write 0xb2ebe00 thru 0xb2fa52b (0xe72c bytes) 2024-09-17T23:30:43.929 INFO:teuthology.orchestra.run.smithi139.stdout:4653 read 0x11e4a00 thru 0x11f064c (0xbc4d bytes) 2024-09-17T23:30:43.930 INFO:teuthology.orchestra.run.smithi139.stdout:4656 write 0xb4ce00 thru 0xb52fa6 (0x61a7 bytes) 2024-09-17T23:30:43.931 INFO:teuthology.orchestra.run.smithi139.stdout:4658 read 0x5598000 thru 0x559b5df (0x35e0 bytes) 2024-09-17T23:30:43.931 INFO:teuthology.orchestra.run.smithi139.stdout:4659 write 0x5afde00 thru 0x5b00e34 (0x3035 bytes) 2024-09-17T23:30:43.932 INFO:teuthology.orchestra.run.smithi139.stdout:4660 punch from 0xa03c000 to 0xa04b600, (0xf600 bytes) 2024-09-17T23:30:43.932 INFO:teuthology.orchestra.run.smithi139.stdout:4662 write 0x4242800 thru 0x424ed86 (0xc587 bytes) 2024-09-17T23:30:43.933 INFO:teuthology.orchestra.run.smithi139.stdout:4663 read 0x6583e00 thru 0x658e2dc (0xa4dd bytes) 2024-09-17T23:30:43.934 INFO:teuthology.orchestra.run.smithi139.stdout:4664 write 0x48b4800 thru 0x48bdbc5 (0x93c6 bytes) 2024-09-17T23:30:43.935 INFO:teuthology.orchestra.run.smithi139.stdout:4665 write 0x9d23200 thru 0x9d29d3b (0x6b3c bytes) 2024-09-17T23:30:43.935 INFO:teuthology.orchestra.run.smithi139.stdout:4666 read 0x30c7a00 thru 0x30ca47d (0x2a7e bytes) 2024-09-17T23:30:43.935 INFO:teuthology.orchestra.run.smithi139.stdout:4669 write 0x175aa00 thru 0x1762291 (0x7892 bytes) 2024-09-17T23:30:43.936 INFO:teuthology.orchestra.run.smithi139.stdout:4670 write 0x3e8fe00 thru 0x3e937db (0x39dc bytes) 2024-09-17T23:30:43.938 INFO:teuthology.orchestra.run.smithi139.stdout:4672 trunc from 0xb2fa600 to 0xbd06e00 2024-09-17T23:30:43.953 INFO:teuthology.orchestra.run.smithi139.stdout:4675 punch from 0x5484600 to 0x5485000, (0xa00 bytes) 2024-09-17T23:30:43.958 INFO:teuthology.orchestra.run.smithi139.stdout:4676 write 0x2de00 thru 0x322a1 (0x44a2 bytes) 2024-09-17T23:30:43.959 INFO:teuthology.orchestra.run.smithi139.stdout:4677 read 0x4b42200 thru 0x4b4c9a9 (0xa7aa bytes) 2024-09-17T23:30:43.960 INFO:teuthology.orchestra.run.smithi139.stdout:4678 read 0xa97d000 thru 0xa988a2d (0xba2e bytes) 2024-09-17T23:30:43.969 INFO:teuthology.orchestra.run.smithi139.stdout:4679 trunc from 0xbd06e00 to 0x3fffa00 2024-09-17T23:30:43.989 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:43.986+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:43.998 INFO:teuthology.orchestra.run.smithi139.stdout:4680 read 0x1783000 thru 0x17880b3 (0x50b4 bytes) 2024-09-17T23:30:43.998 INFO:teuthology.orchestra.run.smithi139.stdout:4685 write 0x3561200 thru 0x35697c4 (0x85c5 bytes) 2024-09-17T23:30:43.999 INFO:teuthology.orchestra.run.smithi139.stdout:4686 read 0xaea600 thru 0xaec8ce (0x22cf bytes) 2024-09-17T23:30:44.000 INFO:teuthology.orchestra.run.smithi139.stdout:4688 read 0x341cc00 thru 0x3423e44 (0x7245 bytes) 2024-09-17T23:30:44.000 INFO:teuthology.orchestra.run.smithi139.stdout:4690 punch from 0x31f0000 to 0x31f3400, (0x3400 bytes) 2024-09-17T23:30:44.000 INFO:teuthology.orchestra.run.smithi139.stdout:4693 read 0x97a600 thru 0x981d9b (0x779c bytes) 2024-09-17T23:30:44.001 INFO:teuthology.orchestra.run.smithi139.stdout:4695 read 0x162c00 thru 0x164df5 (0x21f6 bytes) 2024-09-17T23:30:44.001 INFO:teuthology.orchestra.run.smithi139.stdout:4696 read 0x1dd6400 thru 0x1de053a (0xa13b bytes) 2024-09-17T23:30:44.002 INFO:teuthology.orchestra.run.smithi139.stdout:4697 write 0x14ac600 thru 0x14b6d64 (0xa765 bytes) 2024-09-17T23:30:44.023 INFO:teuthology.orchestra.run.smithi139.stdout:4698 write 0x8eade00 thru 0x8ebbc41 (0xde42 bytes) 2024-09-17T23:30:44.025 INFO:teuthology.orchestra.run.smithi139.stdout:4699 write 0x23bc600 thru 0x23c97a9 (0xd1aa bytes) 2024-09-17T23:30:44.025 INFO:teuthology.orchestra.run.smithi139.stdout:4700 read 0x4f0cc00 thru 0x4f0d3fd (0x7fe bytes) 2024-09-17T23:30:44.032 INFO:teuthology.orchestra.run.smithi139.stdout:4701 trunc from 0x8ebbe00 to 0xe3aa200 2024-09-17T23:30:44.045 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:44.045 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:44.045 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:44.045 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:44.045 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:44.046 INFO:teuthology.orchestra.run.smithi139.stdout:4702 write 0x4a28800 thru 0x4a2e848 (0x6049 bytes) 2024-09-17T23:30:44.049 INFO:teuthology.orchestra.run.smithi139.stdout:4703 trunc from 0xe3aa200 to 0xcd85200 2024-09-17T23:30:44.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:44.061 INFO:teuthology.orchestra.run.smithi139.stdout:4707 read 0x7a46000 thru 0x7a49e0d (0x3e0e bytes) 2024-09-17T23:30:44.062 INFO:teuthology.orchestra.run.smithi139.stdout:4710 punch from 0x50fac00 to 0x5106600, (0xba00 bytes) 2024-09-17T23:30:44.063 INFO:teuthology.orchestra.run.smithi139.stdout:4713 punch from 0x88c9400 to 0x88cb400, (0x2000 bytes) 2024-09-17T23:30:44.063 INFO:teuthology.orchestra.run.smithi139.stdout:4714 punch from 0x6947c00 to 0x694bc00, (0x4000 bytes) 2024-09-17T23:30:44.063 INFO:teuthology.orchestra.run.smithi139.stdout:4716 read 0x20a0000 thru 0x20ab266 (0xb267 bytes) 2024-09-17T23:30:44.067 INFO:teuthology.orchestra.run.smithi139.stdout:4717 punch from 0x2e16200 to 0x2e17200, (0x1000 bytes) 2024-09-17T23:30:44.081 INFO:teuthology.orchestra.run.smithi139.stdout:4720 trunc from 0xcd85200 to 0x12eea00 2024-09-17T23:30:44.090 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:44.088+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:44.117 INFO:teuthology.orchestra.run.smithi139.stdout:4721 read 0x105e400 thru 0x10674f7 (0x90f8 bytes) 2024-09-17T23:30:44.118 INFO:teuthology.orchestra.run.smithi139.stdout:4722 punch from 0xbf8600 to 0xbf8a00, (0x400 bytes) 2024-09-17T23:30:44.123 INFO:teuthology.orchestra.run.smithi139.stdout:4724 trunc from 0x12eea00 to 0x620e800 2024-09-17T23:30:44.139 INFO:teuthology.orchestra.run.smithi139.stdout:4725 read 0x5bb3200 thru 0x5bb8f16 (0x5d17 bytes) 2024-09-17T23:30:44.140 INFO:teuthology.orchestra.run.smithi139.stdout:4726 punch from 0x340fe00 to 0x3413a00, (0x3c00 bytes) 2024-09-17T23:30:44.140 INFO:teuthology.orchestra.run.smithi139.stdout:4727 read 0x102e200 thru 0x10375d3 (0x93d4 bytes) 2024-09-17T23:30:44.151 INFO:teuthology.orchestra.run.smithi139.stdout:4730 trunc from 0x620e800 to 0xe054000 2024-09-17T23:30:44.183 INFO:teuthology.orchestra.run.smithi139.stdout:4731 trunc from 0xe054000 to 0x738a00 2024-09-17T23:30:44.191 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:44.188+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:44.212 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:44.212 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:44.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:44.212 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:44.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.212 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.213 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:44.213 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.333719716000004, 2024-09-17T23:30:44.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:44.213 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.214 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.214 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.214 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:44.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.214 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:44.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:44.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:44.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.318094586000001, 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:44.222 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:44.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:44.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.318056929999997, 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.228 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:44.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:44.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.318039498999994, 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:44.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:44.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.318004568000006, 2024-09-17T23:30:44.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:44.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.317973147999993, 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.244 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:44.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:44.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.317956522000003, 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:44.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.254 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.317934828000006, 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:44.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 77.317923109999995, 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:44.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.358784590999999, 2024-09-17T23:30:44.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:30:44.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:30:44.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354829426999999, 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.273 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:30:44.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:30:44.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354788768999999, 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:30:44.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:44.283 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354632034000002, 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:44.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:30:44.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354529343999999, 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:30:44.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:44.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:30:44.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:30:44.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354493122000001, 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:44.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:30:44.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.299 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354460827, 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:30:44.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:44.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:30:44.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:30:44.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.306 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.354306015999999, 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:30:44.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.354019696000002, 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:30:44.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:30:44.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.317 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.353951035000001, 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:44.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:30:44.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.353909061, 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:30:44.323 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:30:44.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.328 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.329 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:44.329 INFO:teuthology.orchestra.run.smithi139.stdout:4734 clone 31 order 18 su 8192 sc 9 2024-09-17T23:30:44.330 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:44.331 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:44.327+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:44.431 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:44.429+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:44.438 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:44.439 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:44.439 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:44.439 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:44.447 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:44.533 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:44.530+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:44.556 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:44.556 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:44.556 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:44.556 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:44.556 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:44.563 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:44.633 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:44.631+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:44.672 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.175232131000001, 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:30:44.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.175212251000005, 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:30:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.175190713000006, 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.682 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:30:44.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173308114999998, 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:30:44.686 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:30:44.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173297998999999, 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.691 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173285769000003, 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:30:44.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173275200000006, 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:30:44.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:30:44.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:44.706 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173265311999998, 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:30:44.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173255030000007, 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.712 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 67.173239143000004, 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:30:44.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.720 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.634313826000003, 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.721 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:44.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:30:44.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:30:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:30:44.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.632244440000001, 2024-09-17T23:30:44.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:44.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:30:44.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 45.300377072000003, 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:30:44.733 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:30:44.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:30:44.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.815412393999999, 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:30:44.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.815380024, 2024-09-17T23:30:44.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:30:44.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.748 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.815087646999999, 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:30:44.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:30:44.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.753 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.809263509000001, 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:30:44.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.809222005999999, 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:30:44.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.765 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.766483739000002, 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:30:44.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.770 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.766381398, 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:30:44.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:44.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:30:44.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:30:44.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.776 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:44.776 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:44.776 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:44.777 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:44.778 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:44.775+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:44.878 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:44.875+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:44.925 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:44.926 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:44.926 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:44.926 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:44.930 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.12 pg_upmap mapping to [2,1] 2024-09-17T23:30:44.935 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:44.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:30:45.020 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:45.017+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:45.079 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:45.077+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:45.086 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:45.086 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:45.087 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:45.087 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:45.087 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:45.095 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:45.181 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:45.178+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1376 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.493101965000001, 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630741999999999, 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1376 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749803+0000", 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.77e-07 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749805+0000", 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0880000000000002e-06 2024-09-17T23:30:45.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749821+0000", 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5804999999999999e-05 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749827+0000", 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6280000000000002e-06 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776122+0000", 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026295731999999999 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776193+0000", 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0400000000000004e-05 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776433+0000", 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000240712 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.250 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1377 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.493069603000002, 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026847863, 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1377 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749835+0000", 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749836+0000", 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.44e-07 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749838+0000", 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749845+0000", 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7850000000000003e-06 2024-09-17T23:30:45.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749850+0000", 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2249999999999999e-06 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776481+0000", 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026630603999999999 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776572+0000", 2024-09-17T23:30:45.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1186000000000004e-05 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776683+0000", 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011092899999999999 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1378 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:45.255 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.493048420999997, 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027124100000000002, 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1378 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:45.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749856+0000", 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749857+0000", 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8000000000000001e-07 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749864+0000", 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7260000000000002e-06 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749871+0000", 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749876+0000", 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2820000000000004e-06 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776757+0000", 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.026880851000000001 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776877+0000", 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011974200000000001 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.776980+0000", 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000103644 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.259 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1379 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 561152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.493021859000002, 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027251794999999999, 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1379 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749883+0000", 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749885+0000", 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0619999999999999e-06 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749892+0000", 2024-09-17T23:30:45.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9079999999999998e-06 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749898+0000", 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777017+0000", 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027119072000000001 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777063+0000", 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6391999999999999e-05 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777135+0000", 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1580000000000002e-05 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.264 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1380 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 638976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.492999685999997, 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027400681, 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1380 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749905+0000", 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749908+0000", 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749914+0000", 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749918+0000", 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8790000000000002e-06 2024-09-17T23:30:45.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777189+0000", 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027270466 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777237+0000", 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8429e-05 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777306+0000", 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8360000000000003e-05 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1381 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 720896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.492979773999998, 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027804813000000001, 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1381 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749925+0000", 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.27e-07 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749927+0000", 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1019999999999999e-06 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749933+0000", 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749939+0000", 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7389999999999996e-06 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777351+0000", 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027412265000000002 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777443+0000", 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1533999999999995e-05 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777730+0000", 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028694099999999998 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.273 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1382 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 798720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.492933098000002, 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028009221000000001, 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1382 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.274 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749972+0000", 2024-09-17T23:30:45.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749976+0000", 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5389999999999999e-06 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749983+0000", 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3010000000000001e-06 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749989+0000", 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0059999999999998e-06 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777780+0000", 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.027790639999999998 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777892+0000", 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112298 2024-09-17T23:30:45.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.777981+0000", 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8836000000000001e-05 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1383 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.492906349999998, 2024-09-17T23:30:45.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028218073999999999, 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1383 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:45.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749998+0000", 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.749999+0000", 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750006+0000", 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6529999999999994e-06 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750014+0000", 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0480000000000001e-06 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750019+0000", 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3979999999999998e-06 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778022+0000", 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028002697999999999 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778109+0000", 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7014000000000004e-05 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778216+0000", 2024-09-17T23:30:45.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107772 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1384 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 954368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.492878261999998, 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028373809, 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1384 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750026+0000", 2024-09-17T23:30:45.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750027+0000", 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750029+0000", 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2019999999999998e-06 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750035+0000", 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750040+0000", 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778267+0000", 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028227076 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778327+0000", 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9156000000000001e-05 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778400+0000", 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3697999999999998e-05 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4914.0:1385 3.1f 3:f86fc934:::rbd_data.1152ac6484f7.0000000000000008:10 [read 1032192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T23:30:45.287 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.492857962999999, 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028597757000000001, 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4914", 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3321861905", 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1385 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750047+0000", 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.72e-07 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750049+0000", 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.871e-06 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750055+0000", 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3289999999999999e-06 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.750060+0000", 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0000000000000004e-06 2024-09-17T23:30:45.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778452+0000", 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.028391513 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778532+0000", 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0605999999999994e-05 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:49.778644+0000", 2024-09-17T23:30:45.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011206599999999999 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4968.0:12636 3.10 3:0974a81b:::rbd_data.115262cdab6.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.666059435999998, 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.081061670000000002, 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:45.292 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4968", 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12636 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576845+0000", 2024-09-17T23:30:45.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.576847+0000", 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-06 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614529+0000", 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.037681080999999998 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614569+0000", 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.002e-05 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614584+0000", 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5664000000000001e-05 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614790+0000", 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000205828 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.614977+0000", 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018684499999999999 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.616911+0000", 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001933742 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.634692+0000", 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.017780842000000002 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657788+0000", 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.023096390000000001 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657845+0000", 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7497000000000002e-05 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:53.657907+0000", 2024-09-17T23:30:45.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1545999999999997e-05 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3143 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.162109766, 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0144227690000001, 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.298 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3143 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080795+0000", 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080796+0000", 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2099999999999995e-07 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080801+0000", 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6909999999999996e-06 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080822+0000", 2024-09-17T23:30:45.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1128000000000002e-05 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080830+0000", 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8280000000000003e-06 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1292999999999997e-05 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:45.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3889999999999999e-06 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083687+0000", 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027885470000000002 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083696+0000", 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3959999999999995e-06 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095020+0000", 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011324347 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095106+0000", 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5352000000000005e-05 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095218+0000", 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111877 2024-09-17T23:30:45.303 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3144 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.162055219000003, 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014824425, 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3144 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080849+0000", 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080850+0000", 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.75e-07 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080853+0000", 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2509999999999999e-06 2024-09-17T23:30:45.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080861+0000", 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7840000000000004e-06 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080866+0000", 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3730000000000001e-06 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080962+0000", 2024-09-17T23:30:45.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5360999999999997e-05 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4579999999999998e-06 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083738+0000", 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00276998 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083747+0000", 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1139999999999995e-06 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095514+0000", 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011767071 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095579+0000", 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4732000000000003e-05 2024-09-17T23:30:45.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095674+0000", 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4726000000000001e-05 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3145 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.310 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.162029355999998, 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.014984876, 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3145 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080875+0000", 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080876+0000", 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080878+0000", 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1799999999999999e-06 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080886+0000", 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3599999999999996e-06 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080891+0000", 2024-09-17T23:30:45.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8559999999999998e-06 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081004+0000", 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000112568 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081009+0000", 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6699999999999999e-06 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083807+0000", 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002797538 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083825+0000", 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8416000000000001e-05 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095733+0000", 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.011907194 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095780+0000", 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7942000000000003e-05 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095860+0000", 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9721999999999993e-05 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.316 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3146 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.162006638999998, 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015105355, 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3146 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080898+0000", 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7e-07 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080900+0000", 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8679999999999999e-06 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080906+0000", 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7440000000000004e-06 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080911+0000", 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081059+0000", 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014868999999999999 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081067+0000", 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5650000000000004e-06 2024-09-17T23:30:45.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083866+0000", 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002799008 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083871+0000", 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9509999999999997e-06 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095889+0000", 2024-09-17T23:30:45.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012018283 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.095931+0000", 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1885999999999998e-05 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096003+0000", 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2390000000000003e-05 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3147 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.161988508999997, 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015262753, 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3147 2024-09-17T23:30:45.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080916+0000", 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080917+0000", 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6199999999999999e-07 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080918+0000", 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7999999999999999e-06 2024-09-17T23:30:45.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080924+0000", 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5310000000000001e-06 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080928+0000", 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2769999999999999e-06 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081109+0000", 2024-09-17T23:30:45.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018069200000000001 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081115+0000", 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7490000000000004e-06 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083903+0000", 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027881249999999998 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083908+0000", 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0590000000000002e-06 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096026+0000", 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0121184919999999 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096065+0000", 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8566999999999999e-05 2024-09-17T23:30:45.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096179+0000", 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011409900000000001 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3148 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.161952436999997, 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.015338839, 2024-09-17T23:30:45.329 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3148 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080952+0000", 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080953+0000", 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4099999999999999e-07 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080955+0000", 2024-09-17T23:30:45.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.21e-06 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080963+0000", 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7910000000000005e-06 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080968+0000", 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9350000000000002e-06 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081149+0000", 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018104800000000001 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081152+0000", 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.224e-06 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083933+0000", 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027811630000000001 2024-09-17T23:30:45.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083937+0000", 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0609999999999997e-06 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096195+0000", 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122576050000001 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096232+0000", 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7592999999999997e-05 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096291+0000", 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8767999999999997e-05 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3149 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.335 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.161911869000001, 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154068810000001, 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3149 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080993+0000", 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2e-07 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.080995+0000", 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1299999999999999e-06 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081003+0000", 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4089999999999997e-06 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081008+0000", 2024-09-17T23:30:45.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8940000000000001e-06 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081196+0000", 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018834400000000001 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081202+0000", 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9050000000000002e-06 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083961+0000", 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0027589340000000002 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083965+0000", 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0029999999999996e-06 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096305+0000", 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.012340311 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096343+0000", 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7654000000000002e-05 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096400+0000", 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6876999999999998e-05 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5092.0:3150 3.17 3:e8747b13:::rbd_data.1152ce7c866.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.161890845999999, 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0154925990000001, 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5092", 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3966431514", 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 3150 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081014+0000", 2024-09-17T23:30:45.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081016+0000", 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.916e-06 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081022+0000", 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.66e-06 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081026+0000", 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3490000000000002e-06 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081221+0000", 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000194599 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.081223+0000", 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.694e-06 2024-09-17T23:30:45.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083982+0000", 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002758122 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:08.083985+0000", 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9360000000000001e-06 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096413+0000", 2024-09-17T23:30:45.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124277079999999 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096450+0000", 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6449999999999998e-05 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:09.096506+0000", 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6820999999999997e-05 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4997.0:1730 3.1f 3:fa9e12c1:::rbd_data.1152ce7c866.0000000000000117:head [sparse-read 4096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.392394391000003, 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063375800999999995, 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4997", 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1359963294", 2024-09-17T23:30:45.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1730 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850510+0000", 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850511+0000", 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850517+0000", 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2199999999999997e-06 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850522+0000", 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:45.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.850526+0000", 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913749+0000", 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.063222924 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:12.913886+0000", 2024-09-17T23:30:45.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013731999999999999 2024-09-17T23:30:45.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.352 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.352 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:45.352 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:45.352 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:45.353 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T23:30:45.354 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:45.351+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:45.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:45.454 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:45.451+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:45.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:45.474 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:45.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:45.555 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:45.552+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:45.643 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:45.644 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:45.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:45.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:45.656 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:45.653+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:45.757 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:45.754+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:45.773 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:45.805 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:45.805 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:45.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:45.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:45.825 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:45.827 INFO:teuthology.orchestra.run.smithi139.stdout:4735 read 0x6a8c00 thru 0x6ac335 (0x3736 bytes) 2024-09-17T23:30:45.828 INFO:teuthology.orchestra.run.smithi139.stdout:4736 punch from 0x5a0600 to 0x5a0800, (0x200 bytes) 2024-09-17T23:30:45.847 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:45.847 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:45.857 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:45.855+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:45.865 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:45.865 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:45.866 INFO:teuthology.orchestra.run.smithi139.stdout:4737 write 0x6061a00 thru 0x606e74a (0xcd4b bytes) 2024-09-17T23:30:45.866 INFO:teuthology.orchestra.run.smithi139.stdout:4738 read 0x765e00 thru 0x765ee4 (0xe5 bytes) 2024-09-17T23:30:45.866 INFO:teuthology.orchestra.run.smithi139.stdout:4739 read 0x2000e00 thru 0x2005a01 (0x4c02 bytes) 2024-09-17T23:30:45.901 INFO:teuthology.orchestra.run.smithi139.stdout:4740 write 0xdadde00 thru 0xdaebb2b (0xdd2c bytes) 2024-09-17T23:30:45.902 INFO:teuthology.orchestra.run.smithi139.stdout:4741 write 0x6c1a00 thru 0x6c9692 (0x7c93 bytes) 2024-09-17T23:30:45.902 INFO:teuthology.orchestra.run.smithi139.stdout:4743 write 0x3cb3000 thru 0x3cb6d1c (0x3d1d bytes) 2024-09-17T23:30:45.902 INFO:teuthology.orchestra.run.smithi139.stdout:4744 read 0xb3ca000 thru 0xb3ca7c5 (0x7c6 bytes) 2024-09-17T23:30:45.902 INFO:teuthology.orchestra.run.smithi139.stdout:4745 punch from 0x370b600 to 0x370e200, (0x2c00 bytes) 2024-09-17T23:30:45.912 INFO:teuthology.orchestra.run.smithi139.stdout:4746 trunc from 0xdaebc00 to 0x6240e00 2024-09-17T23:30:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:30:45.946 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:45.946 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:45.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:45.958 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:45.956+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:46.060 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:46.057+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:46.103 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:46.103 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:46.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:46.197 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:46.194+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:46.200 INFO:teuthology.orchestra.run.smithi139.stdout:4748 read 0xc11800 thru 0xc15809 (0x400a bytes) 2024-09-17T23:30:46.200 INFO:teuthology.orchestra.run.smithi139.stdout:4752 read 0x4bd000 thru 0x4cbb01 (0xeb02 bytes) 2024-09-17T23:30:46.228 INFO:teuthology.orchestra.run.smithi139.stdout:4753 write 0xc929800 thru 0xc92ed97 (0x5598 bytes) 2024-09-17T23:30:46.228 INFO:teuthology.orchestra.run.smithi139.stdout:4754 write 0xc3aba00 thru 0xc3b9236 (0xd837 bytes) 2024-09-17T23:30:46.228 INFO:teuthology.orchestra.run.smithi139.stdout:4755 write 0x9960e00 thru 0x9970d97 (0xff98 bytes) 2024-09-17T23:30:46.228 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-17T23:30:46.239 INFO:teuthology.orchestra.run.smithi139.stdout:4758 trunc from 0xc92ee00 to 0x3262e00 2024-09-17T23:30:46.261 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:46.258+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:46.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:46.309 INFO:teuthology.orchestra.run.smithi139.stdout:4759 punch from 0x187200 to 0x196200, (0xf000 bytes) 2024-09-17T23:30:46.311 INFO:teuthology.orchestra.run.smithi139.stdout:4761 write 0x27a6e00 thru 0x27ae651 (0x7852 bytes) 2024-09-17T23:30:46.312 INFO:teuthology.orchestra.run.smithi139.stdout:4762 read 0x2f38000 thru 0x2f428bf (0xa8c0 bytes) 2024-09-17T23:30:46.362 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:46.360+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:46.375 INFO:teuthology.orchestra.run.smithi139.stdout:4763 write 0x8bb6600 thru 0x8bc4c13 (0xe614 bytes) 2024-09-17T23:30:46.375 INFO:teuthology.orchestra.run.smithi139.stdout:4764 read 0x705be00 thru 0x706a5f5 (0xe7f6 bytes) 2024-09-17T23:30:46.375 INFO:teuthology.orchestra.run.smithi139.stdout:4767 write 0x3bb4000 thru 0x3bbb26d (0x726e bytes) 2024-09-17T23:30:46.375 INFO:teuthology.orchestra.run.smithi139.stdout:4768 write 0x368c200 thru 0x36950ee (0x8eef bytes) 2024-09-17T23:30:46.375 INFO:teuthology.orchestra.run.smithi139.stdout:4769 read 0x6ef9400 thru 0x6efc71e (0x331f bytes) 2024-09-17T23:30:46.378 INFO:teuthology.orchestra.run.smithi139.stdout:4770 trunc from 0x8bc4e00 to 0x5afb800 2024-09-17T23:30:46.420 INFO:teuthology.orchestra.run.smithi139.stdout:4771 read 0x4843600 thru 0x4846a04 (0x3405 bytes) 2024-09-17T23:30:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:46.429 INFO:teuthology.orchestra.run.smithi139.stdout:4772 trunc from 0x5afb800 to 0xdb48e00 2024-09-17T23:30:46.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:46.444 INFO:teuthology.orchestra.run.smithi139.stdout:4773 write 0xda97a00 thru 0xda99ffd (0x25fe bytes) 2024-09-17T23:30:46.444 INFO:teuthology.orchestra.run.smithi139.stdout:4774 punch from 0xcdd5600 to 0xcde1600, (0xc000 bytes) 2024-09-17T23:30:46.454 INFO:teuthology.orchestra.run.smithi139.stdout:4775 trunc from 0xdb48e00 to 0x5553e00 2024-09-17T23:30:46.464 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:46.461+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:46.479 INFO:teuthology.orchestra.run.smithi139.stdout:4780 punch from 0x4b53e00 to 0x4b54400, (0x600 bytes) 2024-09-17T23:30:46.479 INFO:teuthology.orchestra.run.smithi139.stdout:4781 read 0x1d28400 thru 0x1d2a99f (0x25a0 bytes) 2024-09-17T23:30:46.503 INFO:teuthology.orchestra.run.smithi139.stdout:4783 write 0x7b4e600 thru 0x7b4fcfc (0x16fd bytes) 2024-09-17T23:30:46.504 INFO:teuthology.orchestra.run.smithi139.stdout:4784 write 0x5a2f600 thru 0x5a3f0f5 (0xfaf6 bytes) 2024-09-17T23:30:46.508 INFO:teuthology.orchestra.run.smithi139.stdout:4786 write 0x1080c00 thru 0x1088e5c (0x825d bytes) 2024-09-17T23:30:46.508 INFO:teuthology.orchestra.run.smithi139.stdout:4787 read 0x64dac00 thru 0x64e28ee (0x7cef bytes) 2024-09-17T23:30:46.508 INFO:teuthology.orchestra.run.smithi139.stdout:4788 write 0x6006600 thru 0x6015c96 (0xf697 bytes) 2024-09-17T23:30:46.545 INFO:teuthology.orchestra.run.smithi139.stdout:4790 write 0xb3e8200 thru 0xb3ea69c (0x249d bytes) 2024-09-17T23:30:46.545 INFO:teuthology.orchestra.run.smithi139.stdout:4791 read 0x601c200 thru 0x60284da (0xc2db bytes) 2024-09-17T23:30:46.546 INFO:teuthology.orchestra.run.smithi139.stdout:4792 trunc from 0xb3ea800 to 0xae45600 2024-09-17T23:30:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:46.567 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:46.564+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:46.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:46.570 INFO:teuthology.orchestra.run.smithi139.stdout:4794 read 0x1774400 thru 0x1781fd1 (0xdbd2 bytes) 2024-09-17T23:30:46.570 INFO:teuthology.orchestra.run.smithi139.stdout:4795 read 0x73da400 thru 0x73e549b (0xb09c bytes) 2024-09-17T23:30:46.570 INFO:teuthology.orchestra.run.smithi139.stdout:4797 punch from 0xa985a00 to 0xa993c00, (0xe200 bytes) 2024-09-17T23:30:46.571 INFO:teuthology.orchestra.run.smithi139.stdout:4798 write 0x129600 thru 0x133c1d (0xa61e bytes) 2024-09-17T23:30:46.571 INFO:teuthology.orchestra.run.smithi139.stdout:4801 write 0xa678600 thru 0xa682857 (0xa258 bytes) 2024-09-17T23:30:46.571 INFO:teuthology.orchestra.run.smithi139.stdout:4804 write 0x741d200 thru 0x7429713 (0xc514 bytes) 2024-09-17T23:30:46.572 INFO:teuthology.orchestra.run.smithi139.stdout:4806 read 0x1609600 thru 0x1618c67 (0xf668 bytes) 2024-09-17T23:30:46.608 INFO:teuthology.orchestra.run.smithi139.stdout:4808 write 0xb460000 thru 0xb463733 (0x3734 bytes) 2024-09-17T23:30:46.632 INFO:teuthology.orchestra.run.smithi139.stdout:4810 write 0xc079400 thru 0xc08639e (0xcf9f bytes) 2024-09-17T23:30:46.633 INFO:teuthology.orchestra.run.smithi139.stdout:4812 read 0x1b45e00 thru 0x1b4673d (0x93e bytes) 2024-09-17T23:30:46.634 INFO:teuthology.orchestra.run.smithi139.stdout:4813 write 0x3969000 thru 0x3974758 (0xb759 bytes) 2024-09-17T23:30:46.634 INFO:teuthology.orchestra.run.smithi139.stdout:4816 punch from 0x222f800 to 0x2233800, (0x4000 bytes) 2024-09-17T23:30:46.635 INFO:teuthology.orchestra.run.smithi139.stdout:4817 write 0xb2f6600 thru 0xb2fd94c (0x734d bytes) 2024-09-17T23:30:46.635 INFO:teuthology.orchestra.run.smithi139.stdout:4818 read 0x6117000 thru 0x611b8f5 (0x48f6 bytes) 2024-09-17T23:30:46.636 INFO:teuthology.orchestra.run.smithi139.stdout:4823 write 0x8b8a600 thru 0x8b91b75 (0x7576 bytes) 2024-09-17T23:30:46.667 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:46.664+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:46.677 INFO:teuthology.orchestra.run.smithi139.stdout:4824 write 0xd5f7c00 thru 0xd5fdb56 (0x5f57 bytes) 2024-09-17T23:30:46.677 INFO:teuthology.orchestra.run.smithi139.stdout:4825 write 0x1f8b600 thru 0x1f97157 (0xbb58 bytes) 2024-09-17T23:30:46.677 INFO:teuthology.orchestra.run.smithi139.stdout:4826 write 0x18ac400 thru 0x18ad1a6 (0xda7 bytes) 2024-09-17T23:30:46.677 INFO:teuthology.orchestra.run.smithi139.stdout:4827 punch from 0xc13c00 to 0xc1f400, (0xb800 bytes) 2024-09-17T23:30:46.678 INFO:teuthology.orchestra.run.smithi139.stdout:4828 read 0x64ed600 thru 0x64fa203 (0xcc04 bytes) 2024-09-17T23:30:46.713 INFO:teuthology.orchestra.run.smithi139.stdout:4829 write 0xed29000 thru 0xed3032a (0x732b bytes) 2024-09-17T23:30:46.713 INFO:teuthology.orchestra.run.smithi139.stdout:4834 read 0x7e8a00 thru 0x7f29ba (0x9fbb bytes) 2024-09-17T23:30:46.714 INFO:teuthology.orchestra.run.smithi139.stdout:4835 read 0x5349800 thru 0x534c366 (0x2b67 bytes) 2024-09-17T23:30:46.714 INFO:teuthology.orchestra.run.smithi139.stdout:4836 write 0x27ed800 thru 0x27ee0ec (0x8ed bytes) 2024-09-17T23:30:46.714 INFO:teuthology.orchestra.run.smithi139.stdout:4837 read 0x3bc5200 thru 0x3bc6162 (0xf63 bytes) 2024-09-17T23:30:46.715 INFO:teuthology.orchestra.run.smithi139.stdout:4838 write 0x987de00 thru 0x98800f8 (0x22f9 bytes) 2024-09-17T23:30:46.715 INFO:teuthology.orchestra.run.smithi139.stdout:4839 write 0xce24c00 thru 0xce29acb (0x4ecc bytes) 2024-09-17T23:30:46.715 INFO:teuthology.orchestra.run.smithi139.stdout:4840 write 0x3bac000 thru 0x3bba515 (0xe516 bytes) 2024-09-17T23:30:46.716 INFO:teuthology.orchestra.run.smithi139.stdout:4841 punch from 0x83de00 to 0x842e00, (0x5000 bytes) 2024-09-17T23:30:46.716 INFO:teuthology.orchestra.run.smithi139.stdout:4843 read 0x65ac400 thru 0x65b2f7d (0x6b7e bytes) 2024-09-17T23:30:46.716 INFO:teuthology.orchestra.run.smithi139.stdout:4845 read 0x4871a00 thru 0x4878859 (0x6e5a bytes) 2024-09-17T23:30:46.717 INFO:teuthology.orchestra.run.smithi139.stdout:4846 trunc from 0xed30400 to 0xd36d600 2024-09-17T23:30:46.736 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:46.736 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:46.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:4848 read 0x7f1c400 thru 0x7f203ea (0x3feb bytes) 2024-09-17T23:30:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:4849 write 0x7c8800 thru 0x7ca21b (0x1a1c bytes) 2024-09-17T23:30:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:4851 read 0x98dca00 thru 0x98dd0e0 (0x6e1 bytes) 2024-09-17T23:30:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:4852 read 0x13c9400 thru 0x13d4d56 (0xb957 bytes) 2024-09-17T23:30:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:4853 read 0x8ce9c00 thru 0x8cf85c9 (0xe9ca bytes) 2024-09-17T23:30:46.757 INFO:teuthology.orchestra.run.smithi139.stdout:4859 punch from 0xc98fc00 to 0xc99f600, (0xfa00 bytes) 2024-09-17T23:30:46.758 INFO:teuthology.orchestra.run.smithi139.stdout:4860 read 0x21a7e00 thru 0x21ac2e0 (0x44e1 bytes) 2024-09-17T23:30:46.758 INFO:teuthology.orchestra.run.smithi139.stdout:4862 read 0x2e50a00 thru 0x2e52ca4 (0x22a5 bytes) 2024-09-17T23:30:46.759 INFO:teuthology.orchestra.run.smithi139.stdout:4863 read 0x8f7b200 thru 0x8f80465 (0x5266 bytes) 2024-09-17T23:30:46.759 INFO:teuthology.orchestra.run.smithi139.stdout:4866 write 0x232dc00 thru 0x233317e (0x557f bytes) 2024-09-17T23:30:46.759 INFO:teuthology.orchestra.run.smithi139.stdout:4867 read 0x9551200 thru 0x955ada5 (0x9ba6 bytes) 2024-09-17T23:30:46.759 INFO:teuthology.orchestra.run.smithi139.stdout:4872 punch from 0xceb2600 to 0xcebec00, (0xc600 bytes) 2024-09-17T23:30:46.760 INFO:teuthology.orchestra.run.smithi139.stdout:4873 trunc from 0xd36d600 to 0xc8edc00 2024-09-17T23:30:46.787 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:46.784+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:46.811 INFO:teuthology.orchestra.run.smithi139.stdout:4874 read 0x498a200 thru 0x499472d (0xa52e bytes) 2024-09-17T23:30:46.811 INFO:teuthology.orchestra.run.smithi139.stdout:4876 punch from 0x902c000 to 0x9037600, (0xb600 bytes) 2024-09-17T23:30:46.813 INFO:teuthology.orchestra.run.smithi139.stdout:4879 write 0x6e6ae00 thru 0x6e7296d (0x7b6e bytes) 2024-09-17T23:30:46.813 INFO:teuthology.orchestra.run.smithi139.stdout:4883 read 0x3230a00 thru 0x3235646 (0x4c47 bytes) 2024-09-17T23:30:46.826 INFO:teuthology.orchestra.run.smithi139.stdout:4884 trunc from 0xc8edc00 to 0x1201a00 2024-09-17T23:30:46.868 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:46.866+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:46.890 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:46.890 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:46.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:46.937 INFO:teuthology.orchestra.run.smithi139.stdout:4886 punch from 0xb75e00 to 0xb7de00, (0x8000 bytes) 2024-09-17T23:30:46.937 INFO:teuthology.orchestra.run.smithi139.stdout:skipping zero length punch hole 2024-09-17T23:30:46.947 INFO:teuthology.orchestra.run.smithi139.stdout:4888 trunc from 0x1201a00 to 0xa08be00 2024-09-17T23:30:46.964 INFO:teuthology.orchestra.run.smithi139.stdout:4889 trunc from 0xa08be00 to 0x7921a00 2024-09-17T23:30:46.969 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:46.967+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:46.982 INFO:teuthology.orchestra.run.smithi139.stdout:4891 clone 32 order 22 su 32768 sc 10 2024-09-17T23:30:47.019 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.019 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.027 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:47.070 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:47.068+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:47.135 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.135 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:47.171 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:47.169+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:47.251 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.251 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:47.272 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:47.270+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:47.366 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.366 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.373 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:47.370+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:47.375 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:47.474 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:47.471+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:47.481 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.481 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:30:47.574 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:47.571+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:47.598 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.598 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:30:47.691 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:47.689+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:47.754 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.755 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.762 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:30:47.804 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:47.802+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:47.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:47.877 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:47.874+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:47.910 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:47.910 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:47.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:47.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:30:47.977 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:47.974+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:48.023 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:48.024 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:48.026 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone31 from 0x7921a00 (overlap 0x738a00) to 0x0 2024-09-17T23:30:48.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:48.053 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:48.078 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:48.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4142 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.041714+0000", 2024-09-17T23:30:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.015375987000000001, 2024-09-17T23:30:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015391319000000001, 2024-09-17T23:30:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4142 2024-09-17T23:30:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041714+0000", 2024-09-17T23:30:48.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041714+0000", 2024-09-17T23:30:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041714+0000", 2024-09-17T23:30:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9900000000000001e-07 2024-09-17T23:30:48.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041716+0000", 2024-09-17T23:30:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1469999999999999e-06 2024-09-17T23:30:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041723+0000", 2024-09-17T23:30:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-17T23:30:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041728+0000", 2024-09-17T23:30:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3090000000000003e-06 2024-09-17T23:30:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4189 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042515+0000", 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014574056, 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01462516, 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4189 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042515+0000", 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042515+0000", 2024-09-17T23:30:48.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042516+0000", 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042522+0000", 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.533e-06 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042529+0000", 2024-09-17T23:30:48.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9160000000000004e-06 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042534+0000", 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4230 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043318+0000", 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013771680999999999, 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013852478, 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4230 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043318+0000", 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043318+0000", 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043318+0000", 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043320+0000", 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.052e-06 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043326+0000", 2024-09-17T23:30:48.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1340000000000001e-06 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043331+0000", 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9640000000000002e-06 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4262 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044092+0000", 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012997534, 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013106028, 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4262 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044092+0000", 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044092+0000", 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044092+0000", 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9299999999999999e-07 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044094+0000", 2024-09-17T23:30:48.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044101+0000", 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6699999999999997e-06 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044106+0000", 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0499999999999999e-06 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4295 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044795+0000", 2024-09-17T23:30:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012294731999999999, 2024-09-17T23:30:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012430760000000001, 2024-09-17T23:30:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4295 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044795+0000", 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044795+0000", 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044795+0000", 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.75e-07 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044798+0000", 2024-09-17T23:30:48.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2859999999999998e-06 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044804+0000", 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2260000000000002e-06 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044809+0000", 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0649999999999998e-06 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4358 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046369+0000", 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010720227000000001, 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010905392, 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4358 2024-09-17T23:30:48.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046369+0000", 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046369+0000", 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046370+0000", 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.37e-07 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046373+0000", 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0630000000000002e-06 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046382+0000", 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0620000000000007e-06 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046388+0000", 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0510000000000002e-06 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4403 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048359+0000", 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087309190000000002, 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089713310000000008, 2024-09-17T23:30:48.112 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4403 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048359+0000", 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048359+0000", 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048359+0000", 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6199999999999998e-07 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048361+0000", 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.119e-06 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048368+0000", 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.37e-06 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048373+0000", 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3329999999999996e-06 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4456 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050801+0000", 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0062884780000000001, 2024-09-17T23:30:48.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065776810000000002, 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4456 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050801+0000", 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050801+0000", 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050801+0000", 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2800000000000002e-07 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050804+0000", 2024-09-17T23:30:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-09-17T23:30:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050814+0000", 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8360000000000004e-06 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050826+0000", 2024-09-17T23:30:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2707999999999999e-05 2024-09-17T23:30:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4143 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.041735+0000", 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.015354178, 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015695976, 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4143 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041735+0000", 2024-09-17T23:30:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041735+0000", 2024-09-17T23:30:48.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041736+0000", 2024-09-17T23:30:48.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-17T23:30:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041738+0000", 2024-09-17T23:30:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2510000000000001e-06 2024-09-17T23:30:48.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041744+0000", 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4019999999999999e-06 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041750+0000", 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999999e-06 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4195 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042542+0000", 2024-09-17T23:30:48.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014547621, 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014944133, 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4195 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042542+0000", 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042542+0000", 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042542+0000", 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042545+0000", 2024-09-17T23:30:48.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.176e-06 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042552+0000", 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2189999999999998e-06 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042557+0000", 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.729e-06 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4231 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043338+0000", 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013751619, 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014203408000000001, 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4231 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043338+0000", 2024-09-17T23:30:48.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043338+0000", 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043338+0000", 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-07 2024-09-17T23:30:48.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043341+0000", 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0810000000000001e-06 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043347+0000", 2024-09-17T23:30:48.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.127e-06 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043352+0000", 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4263 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044113+0000", 2024-09-17T23:30:48.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012976135, 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013489021, 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4263 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044113+0000", 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044113+0000", 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044114+0000", 2024-09-17T23:30:48.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4299999999999998e-07 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044116+0000", 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2069999999999998e-06 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044123+0000", 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044128+0000", 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1429999999999997e-06 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4324 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044816+0000", 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01227368, 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012843696999999999, 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4324 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044816+0000", 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044816+0000", 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044816+0000", 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044818+0000", 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0770000000000002e-06 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044826+0000", 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9419999999999995e-06 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044832+0000", 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4359 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046397+0000", 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010692518999999999, 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011291717999999999, 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4359 2024-09-17T23:30:48.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046397+0000", 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046397+0000", 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046398+0000", 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046400+0000", 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046406+0000", 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5819999999999998e-06 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046412+0000", 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4360000000000001e-06 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4404 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048380+0000", 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087095590000000004, 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093611340000000001, 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4404 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048380+0000", 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048380+0000", 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048380+0000", 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048392+0000", 2024-09-17T23:30:48.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1059e-05 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048399+0000", 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5170000000000002e-06 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048404+0000", 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3800000000000002e-06 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4458 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050835+0000", 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.006254323, 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069614769999999998, 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4458 2024-09-17T23:30:48.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050835+0000", 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050835+0000", 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051087+0000", 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025173799999999999 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051091+0000", 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4880000000000001e-06 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051102+0000", 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0681e-05 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051109+0000", 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.063e-06 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4144 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.041756+0000", 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.015333159000000001, 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016096822, 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4144 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041756+0000", 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041756+0000", 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041757+0000", 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9299999999999999e-07 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041759+0000", 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0339999999999999e-06 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041765+0000", 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2859999999999996e-06 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041770+0000", 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999997e-06 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4200 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042579+0000", 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01451013, 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015351717000000001, 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4200 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042579+0000", 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042579+0000", 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042580+0000", 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1699999999999998e-07 2024-09-17T23:30:48.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042583+0000", 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7650000000000002e-06 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042604+0000", 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1015000000000001e-05 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042611+0000", 2024-09-17T23:30:48.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0509999999999999e-06 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4232 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043358+0000", 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013731254, 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014631023999999999, 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4232 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043358+0000", 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043358+0000", 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043359+0000", 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043361+0000", 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0360000000000001e-06 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043367+0000", 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1029999999999998e-06 2024-09-17T23:30:48.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043372+0000", 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8169999999999998e-06 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4264 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044135+0000", 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012954836000000001, 2024-09-17T23:30:48.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013908937, 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4264 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044135+0000", 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044135+0000", 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044135+0000", 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044137+0000", 2024-09-17T23:30:48.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2699999999999999e-06 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044144+0000", 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5089999999999999e-06 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044149+0000", 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4325 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044838+0000", 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012251776000000001, 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013263993999999999, 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4325 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044838+0000", 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044838+0000", 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044838+0000", 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044840+0000", 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0370000000000001e-06 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044846+0000", 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.789e-06 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044851+0000", 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.729e-06 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4360 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046419+0000", 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010670984, 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011737565, 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4360 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046419+0000", 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046419+0000", 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046419+0000", 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046421+0000", 2024-09-17T23:30:48.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046428+0000", 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.348e-06 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046433+0000", 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0300000000000001e-06 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4405 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048412+0000", 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0086772959999999993, 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098048969999999999, 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4405 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048412+0000", 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048412+0000", 2024-09-17T23:30:48.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048413+0000", 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048415+0000", 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048422+0000", 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048429+0000", 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.233e-06 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4459 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.051131+0000", 2024-09-17T23:30:48.179 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0059587379999999999, 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071427069999999999, 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4459 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051131+0000", 2024-09-17T23:30:48.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051131+0000", 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051131+0000", 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9500000000000002e-07 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051134+0000", 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6769999999999999e-06 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051141+0000", 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3690000000000001e-06 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051147+0000", 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4145 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.041777+0000", 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.015312297000000001, 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016556687, 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4145 2024-09-17T23:30:48.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041777+0000", 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041777+0000", 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041778+0000", 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041780+0000", 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041786+0000", 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041791+0000", 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0239999999999997e-06 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4201 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042620+0000", 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014469278, 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015751745000000001, 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4201 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042620+0000", 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042620+0000", 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042621+0000", 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000003e-07 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042623+0000", 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6189999999999998e-06 2024-09-17T23:30:48.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042630+0000", 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7240000000000003e-06 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042635+0000", 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4233 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043380+0000", 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013709902, 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015023296, 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4233 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043380+0000", 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043380+0000", 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043380+0000", 2024-09-17T23:30:48.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043382+0000", 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0820000000000001e-06 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043388+0000", 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043393+0000", 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9389999999999996e-06 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4265 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044155+0000", 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012934222, 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014294351, 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4265 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044155+0000", 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044155+0000", 2024-09-17T23:30:48.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044156+0000", 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044158+0000", 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044164+0000", 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0480000000000004e-06 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044169+0000", 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000002e-06 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4326 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044857+0000", 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012232715, 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013626552, 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4326 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044857+0000", 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044857+0000", 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044857+0000", 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.39e-07 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044859+0000", 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8619999999999999e-06 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044865+0000", 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6910000000000002e-06 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044869+0000", 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6419999999999998e-06 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4361 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046440+0000", 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010649604, 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012100227, 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4361 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046440+0000", 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046440+0000", 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046440+0000", 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3599999999999999e-07 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046443+0000", 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1880000000000001e-06 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046449+0000", 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2330000000000003e-06 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046454+0000", 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7829999999999998e-06 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4408 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048437+0000", 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0086524970000000003, 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010176839999999999, 2024-09-17T23:30:48.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4408 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048437+0000", 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048437+0000", 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048438+0000", 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048440+0000", 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8760000000000001e-06 2024-09-17T23:30:48.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048460+0000", 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9143e-05 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048468+0000", 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9850000000000006e-06 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4460 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.051154+0000", 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0059354830000000001, 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075157490000000004, 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4460 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051154+0000", 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.051154+0000", 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.052567+0000", 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014133069999999999 2024-09-17T23:30:48.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.052572+0000", 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.057e-06 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.052585+0000", 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2723e-05 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.052593+0000", 2024-09-17T23:30:48.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6399999999999997e-06 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4146 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.041800+0000", 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.015289272, 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016929887000000001, 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4146 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041800+0000", 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041800+0000", 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041801+0000", 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6199999999999998e-07 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041803+0000", 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7429999999999998e-06 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041815+0000", 2024-09-17T23:30:48.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1844e-05 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.041821+0000", 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7930000000000003e-06 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4202 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042642+0000", 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014447702999999999, 2024-09-17T23:30:48.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016144451000000001, 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4202 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042642+0000", 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042642+0000", 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042642+0000", 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-07 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042644+0000", 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9700000000000002e-06 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042650+0000", 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1399999999999997e-06 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042655+0000", 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.848e-06 2024-09-17T23:30:48.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4234 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043399+0000", 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013690444, 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015443714000000001, 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4234 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043399+0000", 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043399+0000", 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043400+0000", 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043405+0000", 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7250000000000002e-06 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043412+0000", 2024-09-17T23:30:48.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929999999999996e-06 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043417+0000", 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9400000000000001e-06 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4266 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044175+0000", 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012914103999999999, 2024-09-17T23:30:48.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014732686, 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4266 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044175+0000", 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044175+0000", 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044176+0000", 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044178+0000", 2024-09-17T23:30:48.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.058e-06 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044184+0000", 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0360000000000003e-06 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044189+0000", 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7049999999999998e-06 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4327 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044876+0000", 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012213536000000001, 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014088753000000001, 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4327 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044876+0000", 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044876+0000", 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044876+0000", 2024-09-17T23:30:48.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3099999999999999e-07 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044878+0000", 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8309999999999999e-06 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044884+0000", 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7860000000000002e-06 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044888+0000", 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4362 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046460+0000", 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010629616999999999, 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012560263, 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4362 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046460+0000", 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046460+0000", 2024-09-17T23:30:48.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046460+0000", 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046463+0000", 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.543e-06 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046469+0000", 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3790000000000003e-06 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046474+0000", 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9300000000000002e-06 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4410 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048477+0000", 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0086128439999999997, 2024-09-17T23:30:48.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010689437, 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4410 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048477+0000", 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048477+0000", 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048477+0000", 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2600000000000002e-07 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048480+0000", 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.661e-06 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048487+0000", 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4259999999999997e-06 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048493+0000", 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4203 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042661+0000", 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014428659999999999, 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016560821999999999, 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4203 2024-09-17T23:30:48.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042661+0000", 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042661+0000", 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042661+0000", 2024-09-17T23:30:48.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042663+0000", 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7770000000000001e-06 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042669+0000", 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7189999999999998e-06 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042673+0000", 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6829999999999999e-06 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4235 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043424+0000", 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013665992, 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015846573999999999, 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4235 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043424+0000", 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043424+0000", 2024-09-17T23:30:48.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043424+0000", 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043426+0000", 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.871e-06 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043432+0000", 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8130000000000001e-06 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043436+0000", 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6820000000000002e-06 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4267 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044196+0000", 2024-09-17T23:30:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012893407000000001, 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015121984, 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4267 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044196+0000", 2024-09-17T23:30:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044196+0000", 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044197+0000", 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044199+0000", 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0049999999999999e-06 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044205+0000", 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1070000000000001e-06 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044210+0000", 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1599999999999997e-06 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4329 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044903+0000", 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012186356000000001, 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014467710999999999, 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4329 2024-09-17T23:30:48.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044903+0000", 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044903+0000", 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044904+0000", 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6899999999999997e-07 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044906+0000", 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044913+0000", 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8000000000000001e-06 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044919+0000", 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.507e-06 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4363 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046481+0000", 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010608118999999999, 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012946467999999999, 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4363 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046481+0000", 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046481+0000", 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046482+0000", 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046484+0000", 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2129999999999998e-06 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046491+0000", 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9450000000000004e-06 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046496+0000", 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3319999999999999e-06 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4413 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048500+0000", 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0085897890000000005, 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010984897, 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4413 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048500+0000", 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048500+0000", 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048500+0000", 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048506+0000", 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6590000000000004e-06 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048513+0000", 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048518+0000", 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3140000000000003e-06 2024-09-17T23:30:48.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4204 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042680+0000", 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014409320999999999, 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016864279999999999, 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4204 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042680+0000", 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042680+0000", 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042681+0000", 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3200000000000001e-07 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042682+0000", 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9250000000000002e-06 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042688+0000", 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0209999999999996e-06 2024-09-17T23:30:48.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042693+0000", 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9069999999999998e-06 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4236 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043449+0000", 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013640583, 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016139144000000001, 2024-09-17T23:30:48.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4236 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043449+0000", 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043449+0000", 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043449+0000", 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043452+0000", 2024-09-17T23:30:48.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043458+0000", 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.879e-06 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043463+0000", 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9239999999999998e-06 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4268 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044219+0000", 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012870647000000001, 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015400759, 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4268 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044219+0000", 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044219+0000", 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044219+0000", 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044222+0000", 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1090000000000001e-06 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044228+0000", 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3419999999999996e-06 2024-09-17T23:30:48.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044233+0000", 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9609999999999996e-06 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4330 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044927+0000", 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012162667, 2024-09-17T23:30:48.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014720674, 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4330 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044927+0000", 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044927+0000", 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044927+0000", 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044929+0000", 2024-09-17T23:30:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0959999999999999e-06 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044936+0000", 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044941+0000", 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2889999999999997e-06 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4364 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046503+0000", 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010586488999999999, 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013172194999999999, 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4364 2024-09-17T23:30:48.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046503+0000", 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046503+0000", 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046503+0000", 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046505+0000", 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0379999999999998e-06 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046512+0000", 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046517+0000", 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0719999999999999e-06 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4415 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048525+0000", 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0085645870000000002, 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011179323999999999, 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4415 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048525+0000", 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048525+0000", 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048525+0000", 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048527+0000", 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-06 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048535+0000", 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5689999999999999e-06 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048542+0000", 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6969999999999996e-06 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4205 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042700+0000", 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014389489, 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017031135999999999, 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4205 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042700+0000", 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042700+0000", 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042701+0000", 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042703+0000", 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1110000000000002e-06 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042709+0000", 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2090000000000002e-06 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042714+0000", 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.994e-06 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4237 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043469+0000", 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013620082, 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016296057999999999, 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4237 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043469+0000", 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043469+0000", 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043470+0000", 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1900000000000003e-07 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043472+0000", 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043479+0000", 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9190000000000002e-06 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043485+0000", 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4999999999999999e-06 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4269 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044240+0000", 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012849862, 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015576677000000001, 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4269 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044240+0000", 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044240+0000", 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044240+0000", 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-09-17T23:30:48.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044247+0000", 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044253+0000", 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3759999999999996e-06 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044258+0000", 2024-09-17T23:30:48.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4331 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044948+0000", 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012141984, 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014928343, 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4331 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044948+0000", 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044948+0000", 2024-09-17T23:30:48.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044948+0000", 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044950+0000", 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.934e-06 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044956+0000", 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8209999999999998e-06 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044961+0000", 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8450000000000002e-06 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4365 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.288 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046523+0000", 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010566188000000001, 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013408552000000001, 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4365 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046523+0000", 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046523+0000", 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046524+0000", 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046526+0000", 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046533+0000", 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5579999999999997e-06 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046538+0000", 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4416 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048550+0000", 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0085396010000000008, 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011440687, 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4416 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048550+0000", 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048550+0000", 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048550+0000", 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7599999999999998e-07 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048553+0000", 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.238e-06 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048559+0000", 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3060000000000003e-06 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048573+0000", 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4206 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042722+0000", 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014367042999999999, 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017306754000000001, 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4206 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042722+0000", 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042722+0000", 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042723+0000", 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-07 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042725+0000", 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.232e-06 2024-09-17T23:30:48.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042732+0000", 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5719999999999999e-06 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042737+0000", 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1050000000000003e-06 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4238 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043492+0000", 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013597545000000001, 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016568088000000002, 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4238 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043492+0000", 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043492+0000", 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043492+0000", 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-09-17T23:30:48.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043495+0000", 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4219999999999999e-06 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043501+0000", 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043508+0000", 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-06 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4270 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044264+0000", 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012825489000000001, 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015823489999999999, 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4270 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044264+0000", 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044264+0000", 2024-09-17T23:30:48.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044264+0000", 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4700000000000002e-07 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044266+0000", 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9369999999999998e-06 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044273+0000", 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6270000000000002e-06 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044278+0000", 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7949999999999998e-06 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4366 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046544+0000", 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010545591, 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013572885999999999, 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4366 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046544+0000", 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046544+0000", 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046544+0000", 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4199999999999996e-07 2024-09-17T23:30:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046547+0000", 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046553+0000", 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3459999999999999e-06 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046558+0000", 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0710000000000002e-06 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4419 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048589+0000", 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0085001069999999998, 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01155514, 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4419 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048589+0000", 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048589+0000", 2024-09-17T23:30:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048590+0000", 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.58e-07 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048593+0000", 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7499999999999999e-06 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048600+0000", 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048606+0000", 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4207 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042743+0000", 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014346281000000001, 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017430995000000001, 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4207 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042743+0000", 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042743+0000", 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042744+0000", 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042746+0000", 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.181e-06 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042752+0000", 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0440000000000001e-06 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042757+0000", 2024-09-17T23:30:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000001e-06 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4239 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043515+0000", 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013574685, 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016686743, 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4239 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043515+0000", 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043515+0000", 2024-09-17T23:30:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043515+0000", 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043517+0000", 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.905e-06 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043524+0000", 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4049999999999997e-06 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043529+0000", 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1850000000000003e-06 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4271 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044284+0000", 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012805449, 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015946272000000001, 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4271 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044284+0000", 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044284+0000", 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044284+0000", 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044287+0000", 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0849999999999999e-06 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044292+0000", 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6860000000000003e-06 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044297+0000", 2024-09-17T23:30:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7380000000000002e-06 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4367 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046573+0000", 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010516983000000001, 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013686359, 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4367 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046573+0000", 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046573+0000", 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046573+0000", 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046575+0000", 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3870000000000002e-06 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046582+0000", 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7329999999999997e-06 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046587+0000", 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2140000000000004e-06 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4420 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048636+0000", 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084531269999999995, 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011649629999999999, 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4420 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048636+0000", 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048636+0000", 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048637+0000", 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.99e-07 2024-09-17T23:30:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048640+0000", 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7549999999999999e-06 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048647+0000", 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.593e-06 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048653+0000", 2024-09-17T23:30:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.463e-06 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4208 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042771+0000", 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014318236, 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017552568000000001, 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4208 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042771+0000", 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042771+0000", 2024-09-17T23:30:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042772+0000", 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1200000000000003e-07 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042774+0000", 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.543e-06 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042783+0000", 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460000000000006e-06 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042789+0000", 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.755e-06 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4240 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043535+0000", 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013554188, 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016840298, 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4240 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043535+0000", 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043535+0000", 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043536+0000", 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1199999999999998e-07 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043538+0000", 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2089999999999999e-06 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043545+0000", 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359999999999997e-06 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043550+0000", 2024-09-17T23:30:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.113e-06 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4272 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044303+0000", 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012787016999999999, 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016131361, 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4272 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044303+0000", 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044303+0000", 2024-09-17T23:30:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044303+0000", 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1800000000000002e-07 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044305+0000", 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.874e-06 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044310+0000", 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6250000000000004e-06 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044315+0000", 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.656e-06 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4368 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046595+0000", 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010494679999999999, 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013887105, 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4368 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046595+0000", 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046595+0000", 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046595+0000", 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046597+0000", 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046604+0000", 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1399999999999997e-06 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046608+0000", 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8019999999999999e-06 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4423 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048660+0000", 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084291279999999993, 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011873332, 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4423 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048660+0000", 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048660+0000", 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048661+0000", 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048663+0000", 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048669+0000", 2024-09-17T23:30:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2029999999999997e-06 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048675+0000", 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9120000000000003e-06 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4475 3.c 3:30473601:::rbd_object_map.11528a3e36a5.0000000000000022:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.058503+0000", 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.9985862, 2024-09-17T23:30:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020978809999999998, 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for sub ops", 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4475 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058503+0000", 2024-09-17T23:30:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058503+0000", 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058505+0000", 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2500000000000001e-06 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058517+0000", 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2802999999999999e-05 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058529+0000", 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2094e-05 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058539+0000", 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0909999999999999e-06 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058597+0000", 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8131999999999998e-05 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058677+0000", 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0184000000000003e-05 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.058993+0000", 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031660899999999998 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.060354+0000", 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013609340000000001 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4209 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042796+0000", 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014293083, 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017862747000000002, 2024-09-17T23:30:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4209 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042796+0000", 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042796+0000", 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042797+0000", 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9100000000000004e-07 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042799+0000", 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1610000000000001e-06 2024-09-17T23:30:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042806+0000", 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.438e-06 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042812+0000", 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9469999999999997e-06 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4241 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043557+0000", 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013532702000000001, 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017155581, 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4241 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043557+0000", 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043557+0000", 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043557+0000", 2024-09-17T23:30:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043559+0000", 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.029e-06 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043574+0000", 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4482e-05 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043579+0000", 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5119999999999999e-06 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4273 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044336+0000", 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012753237000000001, 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01643153, 2024-09-17T23:30:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4273 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044336+0000", 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044336+0000", 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044337+0000", 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044339+0000", 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5900000000000002e-06 2024-09-17T23:30:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044348+0000", 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0399999999999993e-06 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044353+0000", 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5690000000000004e-06 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4369 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046621+0000", 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010468998, 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014186399000000001, 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4369 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046621+0000", 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046621+0000", 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046621+0000", 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-07 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046624+0000", 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046631+0000", 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046636+0000", 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1719999999999998e-06 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4424 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048682+0000", 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0084074070000000004, 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012153212, 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4424 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048682+0000", 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048682+0000", 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048683+0000", 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048684+0000", 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8109999999999999e-06 2024-09-17T23:30:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048691+0000", 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2999999999999998e-06 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048696+0000", 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9450000000000001e-06 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4210 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042820+0000", 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014269364999999999, 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018044422000000001, 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4210 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042820+0000", 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042820+0000", 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042821+0000", 2024-09-17T23:30:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3099999999999998e-07 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042833+0000", 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.225e-05 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042841+0000", 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6009999999999997e-06 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042846+0000", 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.361e-06 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4242 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043587+0000", 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013502482999999999, 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017305431999999999, 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4242 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043587+0000", 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043587+0000", 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043588+0000", 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.58e-07 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043590+0000", 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3999999999999999e-06 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043596+0000", 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1979999999999998e-06 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043601+0000", 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4274 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044361+0000", 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012728468999999999, 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016559001, 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4274 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044361+0000", 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044361+0000", 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044362+0000", 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044364+0000", 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0839999999999999e-06 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044370+0000", 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0440000000000001e-06 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044375+0000", 2024-09-17T23:30:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4370 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046643+0000", 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010446100999999999, 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014305014, 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4370 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046643+0000", 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046643+0000", 2024-09-17T23:30:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046644+0000", 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046646+0000", 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.283e-06 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046655+0000", 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4600000000000003e-06 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046660+0000", 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2900000000000002e-06 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4426 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048702+0000", 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083877589999999998, 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012273657, 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4426 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048702+0000", 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048702+0000", 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048702+0000", 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5199999999999998e-07 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048704+0000", 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9530000000000002e-06 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048710+0000", 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1759999999999998e-06 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048716+0000", 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4269999999999999e-06 2024-09-17T23:30:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4211 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042856+0000", 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014233261000000001, 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018146359000000001, 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4211 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042856+0000", 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042856+0000", 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042857+0000", 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8400000000000005e-07 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042859+0000", 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5050000000000002e-06 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042866+0000", 2024-09-17T23:30:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5570000000000001e-06 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042871+0000", 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3120000000000001e-06 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4243 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043608+0000", 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013481943, 2024-09-17T23:30:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017422191, 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4243 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043608+0000", 2024-09-17T23:30:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043608+0000", 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043608+0000", 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043610+0000", 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043616+0000", 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3450000000000002e-06 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043622+0000", 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1000000000000003e-06 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4275 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044381+0000", 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012708044999999999, 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016675460999999999, 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4275 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044381+0000", 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044381+0000", 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044382+0000", 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4999999999999998e-07 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044384+0000", 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1270000000000001e-06 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044390+0000", 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.302e-06 2024-09-17T23:30:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044395+0000", 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4371 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046667+0000", 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010422760999999999, 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014417338, 2024-09-17T23:30:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4371 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046667+0000", 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046667+0000", 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046667+0000", 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046669+0000", 2024-09-17T23:30:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1600000000000001e-06 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046676+0000", 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1619999999999996e-06 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046682+0000", 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4427 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048723+0000", 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083670240000000007, 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012388668, 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4427 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048723+0000", 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048723+0000", 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048723+0000", 2024-09-17T23:30:48.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.65e-07 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048725+0000", 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2910000000000002e-06 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048731+0000", 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.224e-06 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048736+0000", 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9520000000000002e-06 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4212 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042886+0000", 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014203861999999999, 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018264141000000001, 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4212 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042886+0000", 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042886+0000", 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042886+0000", 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1999999999999999e-07 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042889+0000", 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042896+0000", 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042901+0000", 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1240000000000004e-06 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4244 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043643+0000", 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013446298000000001, 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017535416000000002, 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4244 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043643+0000", 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043643+0000", 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043644+0000", 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2799999999999996e-07 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043646+0000", 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1059999999999998e-06 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043652+0000", 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043659+0000", 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5409999999999997e-06 2024-09-17T23:30:48.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4276 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044402+0000", 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012687878, 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016805765, 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4276 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044402+0000", 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044402+0000", 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044402+0000", 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044404+0000", 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0109999999999999e-06 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044410+0000", 2024-09-17T23:30:48.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.964e-06 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044415+0000", 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4372 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046690+0000", 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01039907, 2024-09-17T23:30:48.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014571601, 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4372 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046690+0000", 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046690+0000", 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046691+0000", 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.32e-07 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046693+0000", 2024-09-17T23:30:48.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3649999999999998e-06 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046702+0000", 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3690000000000007e-06 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046707+0000", 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4759999999999997e-06 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4429 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048750+0000", 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083390689999999993, 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569439999999999, 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4429 2024-09-17T23:30:48.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048750+0000", 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048750+0000", 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048751+0000", 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048753+0000", 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.154e-06 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048760+0000", 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8820000000000003e-06 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048767+0000", 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5409999999999997e-06 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4213 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042909+0000", 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014180122999999999, 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018466844999999999, 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4213 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042909+0000", 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042909+0000", 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042910+0000", 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7100000000000002e-07 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042912+0000", 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042919+0000", 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7440000000000001e-06 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042926+0000", 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8859999999999998e-06 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4245 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043665+0000", 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013424106, 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017767056999999999, 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4245 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043665+0000", 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043665+0000", 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043666+0000", 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1800000000000001e-07 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043668+0000", 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8700000000000001e-06 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043674+0000", 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4790000000000001e-06 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043679+0000", 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-09-17T23:30:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4277 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044422+0000", 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012667837, 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017069368000000001, 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4277 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044422+0000", 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044422+0000", 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044422+0000", 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6899999999999998e-07 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044424+0000", 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1900000000000002e-06 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044431+0000", 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1689999999999997e-06 2024-09-17T23:30:48.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044435+0000", 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9300000000000002e-06 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4340 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.045891+0000", 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011198131, 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015657025000000001, 2024-09-17T23:30:48.431 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4340 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045891+0000", 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045891+0000", 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045892+0000", 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.012e-06 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045897+0000", 2024-09-17T23:30:48.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6310000000000002e-06 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045909+0000", 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1735e-05 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045916+0000", 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4050000000000003e-06 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4376 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046715+0000", 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010374791, 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014895429, 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4376 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046715+0000", 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046715+0000", 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046715+0000", 2024-09-17T23:30:48.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046717+0000", 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2450000000000001e-06 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046725+0000", 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7300000000000005e-06 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046730+0000", 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4431 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048774+0000", 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0083157600000000002, 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012891856, 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4431 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048774+0000", 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048774+0000", 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048774+0000", 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048776+0000", 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2050000000000001e-06 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048783+0000", 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2339999999999999e-06 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048788+0000", 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9660000000000004e-06 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4478 3.c 3:30473601:::rbd_object_map.11528a3e36a5.0000000000000022:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.441 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.061165+0000", 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4294967295.995924, 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00056296299999999998, 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for sub ops", 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4478 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061165+0000", 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061165+0000", 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061166+0000", 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1289999999999999e-06 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061171+0000", 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8579999999999999e-06 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061184+0000", 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2478e-05 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061191+0000", 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.593e-06 2024-09-17T23:30:48.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061239+0000", 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7614000000000002e-05 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061314+0000", 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4507999999999999e-05 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.061622+0000", 2024-09-17T23:30:48.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030864199999999998 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4214 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042933+0000", 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014156568, 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018856643999999999, 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4214 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042933+0000", 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042933+0000", 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042933+0000", 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042936+0000", 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0899999999999999e-06 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042942+0000", 2024-09-17T23:30:48.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4540000000000004e-06 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042947+0000", 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0950000000000003e-06 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4246 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043685+0000", 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013404154, 2024-09-17T23:30:48.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018159947999999999, 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4246 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043685+0000", 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043685+0000", 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043686+0000", 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4700000000000002e-07 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043692+0000", 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.618e-06 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043700+0000", 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043705+0000", 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2719999999999997e-06 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4278 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044444+0000", 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012645596, 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017451668, 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4278 2024-09-17T23:30:48.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044444+0000", 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044444+0000", 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044444+0000", 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044446+0000", 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1289999999999999e-06 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044453+0000", 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1480000000000003e-06 2024-09-17T23:30:48.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044459+0000", 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.117e-06 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4341 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.045966+0000", 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011123981, 2024-09-17T23:30:48.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015978625999999999, 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4341 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045966+0000", 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045966+0000", 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045966+0000", 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4000000000000001e-07 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045969+0000", 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0809999999999998e-06 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045979+0000", 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2709999999999998e-06 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045986+0000", 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.002e-06 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4377 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046737+0000", 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010352273, 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015256204000000001, 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4377 2024-09-17T23:30:48.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046737+0000", 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046737+0000", 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046738+0000", 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046740+0000", 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.131e-06 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046754+0000", 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.398e-05 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046760+0000", 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9159999999999998e-06 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4433 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048794+0000", 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082956709999999993, 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013254905000000001, 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4433 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048794+0000", 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048794+0000", 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048794+0000", 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7700000000000001e-07 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048796+0000", 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048802+0000", 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1920000000000002e-06 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048808+0000", 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4215 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042954+0000", 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014135449, 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019153013, 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4215 2024-09-17T23:30:48.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042954+0000", 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042954+0000", 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042955+0000", 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042957+0000", 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042965+0000", 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9349999999999994e-06 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042970+0000", 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4247 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043713+0000", 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013376647, 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018449686, 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4247 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043713+0000", 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043713+0000", 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043713+0000", 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.89e-07 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043716+0000", 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4040000000000002e-06 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043723+0000", 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9700000000000002e-06 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043728+0000", 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.463e-06 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4279 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044465+0000", 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012624185, 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017752094999999999, 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4279 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044465+0000", 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044465+0000", 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044466+0000", 2024-09-17T23:30:48.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044468+0000", 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0719999999999998e-06 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044474+0000", 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9100000000000002e-06 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044478+0000", 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6999999999999999e-06 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4342 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.045996+0000", 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011093762, 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016256754000000002, 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4342 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045996+0000", 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045996+0000", 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045996+0000", 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2300000000000001e-07 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.045999+0000", 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046007+0000", 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9549999999999992e-06 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046012+0000", 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5740000000000003e-06 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4378 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046767+0000", 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010322095999999999, 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015512924000000001, 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4378 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046767+0000", 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046767+0000", 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046768+0000", 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5499999999999998e-07 2024-09-17T23:30:48.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046770+0000", 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.576e-06 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046777+0000", 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046783+0000", 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.127e-06 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4434 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048814+0000", 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082750510000000003, 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013493554, 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4434 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048814+0000", 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048814+0000", 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048815+0000", 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048817+0000", 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3489999999999999e-06 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048823+0000", 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9630000000000003e-06 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048829+0000", 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4216 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042976+0000", 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014113285999999999, 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019359172000000001, 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4216 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042976+0000", 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042976+0000", 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042977+0000", 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999999e-07 2024-09-17T23:30:48.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042979+0000", 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2479999999999999e-06 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042985+0000", 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042990+0000", 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-06 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4248 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043754+0000", 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013335321000000001, 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018608560999999999, 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4248 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043754+0000", 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043754+0000", 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043755+0000", 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3e-07 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043757+0000", 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043765+0000", 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4660000000000002e-06 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043770+0000", 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.48e-06 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4280 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044484+0000", 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012605531999999999, 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017907567999999999, 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4280 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044484+0000", 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044484+0000", 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044484+0000", 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3700000000000001e-07 2024-09-17T23:30:48.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044486+0000", 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8300000000000001e-06 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044492+0000", 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7100000000000004e-06 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044497+0000", 2024-09-17T23:30:48.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7040000000000002e-06 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4343 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046021+0000", 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011068741999999999, 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016398630000000001, 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4343 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046021+0000", 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046021+0000", 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046021+0000", 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046024+0000", 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.52e-06 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046031+0000", 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0659999999999998e-06 2024-09-17T23:30:48.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046037+0000", 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5300000000000002e-06 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4379 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046791+0000", 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010298754, 2024-09-17T23:30:48.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015664005000000002, 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4379 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046791+0000", 2024-09-17T23:30:48.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046791+0000", 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046791+0000", 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046794+0000", 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4389999999999999e-06 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046800+0000", 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4370000000000004e-06 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046805+0000", 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7550000000000002e-06 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4435 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048836+0000", 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082536999999999992, 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01364713, 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4435 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048836+0000", 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048836+0000", 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048836+0000", 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.96e-07 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048838+0000", 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048845+0000", 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048851+0000", 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6950000000000003e-06 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4217 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.042998+0000", 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014091974, 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019520327000000001, 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4217 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042998+0000", 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042998+0000", 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.042998+0000", 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043000+0000", 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3470000000000001e-06 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043007+0000", 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5719999999999999e-06 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043012+0000", 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999999e-06 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4249 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043778+0000", 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013311517, 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018809174000000001, 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4249 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043778+0000", 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043778+0000", 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043779+0000", 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5599999999999995e-07 2024-09-17T23:30:48.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043781+0000", 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1849999999999998e-06 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043788+0000", 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7499999999999997e-06 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043793+0000", 2024-09-17T23:30:48.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1549999999999998e-06 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4281 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044508+0000", 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012581208, 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018108505, 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4281 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044508+0000", 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044508+0000", 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044509+0000", 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2200000000000004e-07 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044515+0000", 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7200000000000003e-06 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044521+0000", 2024-09-17T23:30:48.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4230000000000002e-06 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044526+0000", 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7849999999999999e-06 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4344 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046045+0000", 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011044852000000001, 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016613063000000001, 2024-09-17T23:30:48.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4344 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046045+0000", 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046045+0000", 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046045+0000", 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4900000000000001e-07 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046052+0000", 2024-09-17T23:30:48.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7009999999999999e-06 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046059+0000", 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2470000000000002e-06 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046066+0000", 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4380 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046812+0000", 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01027786, 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874031, 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4380 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.522 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046812+0000", 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046812+0000", 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046812+0000", 2024-09-17T23:30:48.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046814+0000", 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2670000000000001e-06 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046821+0000", 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2920000000000001e-06 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046826+0000", 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4437 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048866+0000", 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0082234219999999993, 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013846542, 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4437 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048866+0000", 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048866+0000", 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048867+0000", 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999995e-07 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048877+0000", 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9369999999999999e-06 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048885+0000", 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5669999999999995e-06 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048891+0000", 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.507e-06 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4218 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043018+0000", 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014071185999999999, 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019725626, 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4218 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043018+0000", 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043018+0000", 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043019+0000", 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043021+0000", 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043027+0000", 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1040000000000003e-06 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043032+0000", 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9230000000000001e-06 2024-09-17T23:30:48.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4250 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043800+0000", 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013289997, 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018999927, 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4250 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043800+0000", 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043800+0000", 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043800+0000", 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043802+0000", 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0849999999999999e-06 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043810+0000", 2024-09-17T23:30:48.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043816+0000", 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8210000000000004e-06 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4282 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044532+0000", 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012557349000000001, 2024-09-17T23:30:48.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018324699, 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4282 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044532+0000", 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044532+0000", 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044533+0000", 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2699999999999999e-07 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044535+0000", 2024-09-17T23:30:48.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.074e-06 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044541+0000", 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1979999999999998e-06 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044546+0000", 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4345 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046073+0000", 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.011016735, 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016830146000000001, 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4345 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046073+0000", 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046073+0000", 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046073+0000", 2024-09-17T23:30:48.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046075+0000", 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0980000000000001e-06 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046082+0000", 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4269999999999996e-06 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046088+0000", 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3879999999999997e-06 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4381 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046832+0000", 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010257051, 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016099215, 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4381 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046832+0000", 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046832+0000", 2024-09-17T23:30:48.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046833+0000", 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9400000000000001e-07 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046835+0000", 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046841+0000", 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0530000000000003e-06 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046846+0000", 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4439 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048898+0000", 2024-09-17T23:30:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081911610000000006, 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014079071, 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4439 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048898+0000", 2024-09-17T23:30:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048898+0000", 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048899+0000", 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0800000000000004e-07 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048901+0000", 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048908+0000", 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4529999999999999e-06 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048913+0000", 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1850000000000003e-06 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4219 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043039+0000", 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014050899, 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020044597000000001, 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4219 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043039+0000", 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043039+0000", 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043039+0000", 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043041+0000", 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1629999999999999e-06 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043047+0000", 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1229999999999997e-06 2024-09-17T23:30:48.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043052+0000", 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0089999999999998e-06 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4251 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043823+0000", 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013266116, 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019317501000000001, 2024-09-17T23:30:48.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4251 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043823+0000", 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043823+0000", 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043824+0000", 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043826+0000", 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1150000000000001e-06 2024-09-17T23:30:48.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043832+0000", 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2249999999999997e-06 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043837+0000", 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7149999999999997e-06 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4283 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044553+0000", 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012536468, 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018646000999999999, 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4283 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044553+0000", 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044553+0000", 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044554+0000", 2024-09-17T23:30:48.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044556+0000", 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0650000000000001e-06 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044569+0000", 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3461e-05 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044574+0000", 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3920000000000002e-06 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4346 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046097+0000", 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010992132999999999, 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017150757999999999, 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4346 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046097+0000", 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046097+0000", 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046098+0000", 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046100+0000", 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3290000000000001e-06 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046108+0000", 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6120000000000001e-06 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046113+0000", 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4659999999999998e-06 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4382 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046853+0000", 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010236563000000001, 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016423816000000001, 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4382 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046853+0000", 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046853+0000", 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046854+0000", 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.99e-07 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046856+0000", 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1339999999999998e-06 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046862+0000", 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4389999999999997e-06 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046867+0000", 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9300000000000002e-06 2024-09-17T23:30:48.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4441 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048919+0000", 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081702609999999998, 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014384668, 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4441 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048919+0000", 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048919+0000", 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048920+0000", 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048922+0000", 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9970000000000001e-06 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048928+0000", 2024-09-17T23:30:48.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.241e-06 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048933+0000", 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9180000000000002e-06 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4220 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043070+0000", 2024-09-17T23:30:48.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.014019185, 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020395396999999999, 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4220 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043070+0000", 2024-09-17T23:30:48.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043070+0000", 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043071+0000", 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9400000000000005e-07 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043073+0000", 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5500000000000001e-06 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043080+0000", 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8859999999999998e-06 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043086+0000", 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3859999999999998e-06 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4252 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043843+0000", 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013246058, 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019661198000000001, 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4252 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043843+0000", 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043843+0000", 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043844+0000", 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3999999999999997e-07 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043846+0000", 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.835e-06 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043851+0000", 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7729999999999996e-06 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043856+0000", 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7439999999999998e-06 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4285 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044584+0000", 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012505247000000001, 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018998426999999998, 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4285 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044584+0000", 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044584+0000", 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044585+0000", 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044587+0000", 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-09-17T23:30:48.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044594+0000", 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6749999999999996e-06 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044599+0000", 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2109999999999997e-06 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4348 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046121+0000", 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010968612000000001, 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017519395, 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4348 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046121+0000", 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046121+0000", 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046121+0000", 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:30:48.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046124+0000", 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.379e-06 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046130+0000", 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4690000000000002e-06 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046135+0000", 2024-09-17T23:30:48.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0409999999999997e-06 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4383 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046873+0000", 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010216266, 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016826052000000001, 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4383 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046873+0000", 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046873+0000", 2024-09-17T23:30:48.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046874+0000", 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3099999999999999e-07 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046881+0000", 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.922e-06 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046888+0000", 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1980000000000004e-06 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046893+0000", 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9400000000000001e-06 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4443 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048939+0000", 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0081504449999999992, 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014803424000000001, 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4443 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048939+0000", 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048939+0000", 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.049024+0000", 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5319000000000001e-05 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.049028+0000", 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.349e-06 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.049036+0000", 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5280000000000003e-06 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.049042+0000", 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1099999999999999e-06 2024-09-17T23:30:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4221 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043093+0000", 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013996740000000001, 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020678106000000002, 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4221 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043093+0000", 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043093+0000", 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043093+0000", 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2300000000000002e-07 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043095+0000", 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9999999999999999e-06 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043103+0000", 2024-09-17T23:30:48.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.909e-06 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043109+0000", 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2689999999999996e-06 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4253 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043862+0000", 2024-09-17T23:30:48.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013227793, 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019943019999999999, 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4253 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043862+0000", 2024-09-17T23:30:48.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043862+0000", 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043862+0000", 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3599999999999999e-07 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043864+0000", 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6470000000000001e-06 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043871+0000", 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4490000000000002e-06 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043876+0000", 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4286 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044605+0000", 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012484291, 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019226921000000001, 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4286 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044605+0000", 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044605+0000", 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044606+0000", 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0100000000000005e-07 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044608+0000", 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8980000000000001e-06 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044613+0000", 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7790000000000001e-06 2024-09-17T23:30:48.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044618+0000", 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4349 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046142+0000", 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010947531999999999, 2024-09-17T23:30:48.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017719585, 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4349 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046142+0000", 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046142+0000", 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046143+0000", 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9100000000000004e-07 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046145+0000", 2024-09-17T23:30:48.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0499999999999999e-06 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046151+0000", 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3779999999999998e-06 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046156+0000", 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3109999999999996e-06 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4384 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046898+0000", 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010191044, 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016990447999999998, 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4384 2024-09-17T23:30:48.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046898+0000", 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046898+0000", 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046899+0000", 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3700000000000001e-07 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046901+0000", 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.891e-06 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046907+0000", 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046912+0000", 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.724e-06 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4445 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.049053+0000", 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0080370070000000005, 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014867059, 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4445 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.049053+0000", 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.049053+0000", 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050571+0000", 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015187169999999999 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050576+0000", 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9590000000000003e-06 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050589+0000", 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2533e-05 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050599+0000", 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9369999999999999e-06 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4222 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043116+0000", 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013973386000000001, 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020835349, 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4222 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043116+0000", 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043116+0000", 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043117+0000", 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4700000000000002e-07 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043123+0000", 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043129+0000", 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4359999999999999e-06 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043134+0000", 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8500000000000002e-06 2024-09-17T23:30:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4254 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043882+0000", 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01320781, 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020126031999999999, 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4254 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043882+0000", 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043882+0000", 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043882+0000", 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4299999999999999e-07 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043884+0000", 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.874e-06 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043890+0000", 2024-09-17T23:30:48.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043894+0000", 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6149999999999999e-06 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4287 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044624+0000", 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012465611999999999, 2024-09-17T23:30:48.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019440246000000001, 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4287 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044624+0000", 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044624+0000", 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044624+0000", 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999999e-07 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044626+0000", 2024-09-17T23:30:48.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9429999999999999e-06 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044632+0000", 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7060000000000001e-06 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044637+0000", 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7310000000000001e-06 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4350 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046163+0000", 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01092655, 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017960721999999998, 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4350 2024-09-17T23:30:48.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046163+0000", 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046163+0000", 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046163+0000", 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046166+0000", 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046172+0000", 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046177+0000", 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2630000000000003e-06 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4385 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046917+0000", 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010172206, 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017253329000000001, 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4385 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046917+0000", 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046917+0000", 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046918+0000", 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7300000000000002e-07 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046920+0000", 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.917e-06 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046925+0000", 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046930+0000", 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3529999999999997e-06 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.627 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4447 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050616+0000", 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0064739469999999999, 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013586775000000001, 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4447 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050616+0000", 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050616+0000", 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050616+0000", 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999996e-07 2024-09-17T23:30:48.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050619+0000", 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9569999999999998e-06 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050627+0000", 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7800000000000001e-06 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050633+0000", 2024-09-17T23:30:48.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.852e-06 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4223 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043162+0000", 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013927118, 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021067756999999999, 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4223 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043162+0000", 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043162+0000", 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043163+0000", 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1600000000000001e-07 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043165+0000", 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043172+0000", 2024-09-17T23:30:48.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043177+0000", 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-06 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4255 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043901+0000", 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013188218, 2024-09-17T23:30:48.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020359417000000001, 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4255 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043901+0000", 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043901+0000", 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043902+0000", 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043904+0000", 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1619999999999998e-06 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043910+0000", 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9970000000000004e-06 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043915+0000", 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9409999999999998e-06 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4288 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044642+0000", 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012447593999999999, 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019650784000000001, 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4288 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044642+0000", 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044642+0000", 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044642+0000", 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1300000000000001e-07 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044644+0000", 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.863e-06 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044650+0000", 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.767e-06 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044655+0000", 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5870000000000003e-06 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4351 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046186+0000", 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010903892, 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018160989999999998, 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4351 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046186+0000", 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046186+0000", 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046186+0000", 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0699999999999997e-07 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046188+0000", 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2299999999999998e-06 2024-09-17T23:30:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046195+0000", 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5209999999999999e-06 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046200+0000", 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4386 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046936+0000", 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010153796, 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017473381999999999, 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4386 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046936+0000", 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046936+0000", 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046936+0000", 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-09-17T23:30:48.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046938+0000", 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046944+0000", 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2489999999999998e-06 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046949+0000", 2024-09-17T23:30:48.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7559999999999999e-06 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4448 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050641+0000", 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0064484599999999996, 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013823019000000001, 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4448 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050641+0000", 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050641+0000", 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050642+0000", 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7199999999999999e-07 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050644+0000", 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.571e-06 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050651+0000", 2024-09-17T23:30:48.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050656+0000", 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4224 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043184+0000", 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013905832999999999, 2024-09-17T23:30:48.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021339903, 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4224 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043184+0000", 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043184+0000", 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043184+0000", 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999999e-07 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043186+0000", 2024-09-17T23:30:48.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9089999999999998e-06 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043192+0000", 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8209999999999998e-06 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043197+0000", 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.763e-06 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4256 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043944+0000", 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013145670999999999, 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020640163999999999, 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4256 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043944+0000", 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043944+0000", 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043944+0000", 2024-09-17T23:30:48.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-07 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043947+0000", 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5770000000000001e-06 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043955+0000", 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1990000000000006e-06 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043961+0000", 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9100000000000002e-06 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4289 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044660+0000", 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012429572, 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019982875000000001, 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4289 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044660+0000", 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044660+0000", 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044660+0000", 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.39e-07 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044662+0000", 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.012e-06 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044668+0000", 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044673+0000", 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4352 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046206+0000", 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010883446999999999, 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018492023, 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4352 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046206+0000", 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046206+0000", 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046206+0000", 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046209+0000", 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0439999999999998e-06 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046215+0000", 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1820000000000003e-06 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046220+0000", 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0520000000000001e-06 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4387 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046956+0000", 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010133506, 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017803122000000001, 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4387 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046956+0000", 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046956+0000", 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046956+0000", 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4400000000000001e-07 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046959+0000", 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046965+0000", 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6390000000000003e-06 2024-09-17T23:30:48.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046971+0000", 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4450 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050663+0000", 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0064264980000000001, 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014148707999999999, 2024-09-17T23:30:48.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4450 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050663+0000", 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050663+0000", 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050663+0000", 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0600000000000001e-07 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050666+0000", 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.266e-06 2024-09-17T23:30:48.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050672+0000", 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1290000000000001e-06 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050677+0000", 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4225 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043203+0000", 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013886246999999999, 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021663042, 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4225 2024-09-17T23:30:48.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043203+0000", 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043203+0000", 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043204+0000", 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043206+0000", 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2139999999999999e-06 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043212+0000", 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043217+0000", 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0250000000000002e-06 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4257 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043969+0000", 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.01312048, 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020951279, 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4257 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043969+0000", 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043969+0000", 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043970+0000", 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.68e-07 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043972+0000", 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043978+0000", 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3799999999999999e-06 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043984+0000", 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0900000000000001e-06 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4290 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044681+0000", 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012408841, 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020298904999999999, 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4290 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044681+0000", 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044681+0000", 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044681+0000", 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:48.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044683+0000", 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3769999999999999e-06 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044690+0000", 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1029999999999998e-06 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044695+0000", 2024-09-17T23:30:48.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4353 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046226+0000", 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010863097, 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018809689000000001, 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4353 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046226+0000", 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046226+0000", 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046227+0000", 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046229+0000", 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3499999999999999e-06 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046236+0000", 2024-09-17T23:30:48.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7039999999999997e-06 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046241+0000", 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2580000000000003e-06 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4388 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046991+0000", 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010098787, 2024-09-17T23:30:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018103794999999999, 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4388 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046991+0000", 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046991+0000", 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046991+0000", 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6100000000000001e-07 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046994+0000", 2024-09-17T23:30:48.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7690000000000001e-06 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.047001+0000", 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2899999999999997e-06 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.047007+0000", 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6879999999999996e-06 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4451 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050684+0000", 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0064057879999999999, 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014467025999999999, 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4451 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050684+0000", 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050684+0000", 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050684+0000", 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8500000000000002e-07 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050686+0000", 2024-09-17T23:30:48.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0600000000000002e-06 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050692+0000", 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0909999999999998e-06 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050697+0000", 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4226 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043228+0000", 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013861696999999999, 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021978332, 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4226 2024-09-17T23:30:48.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043228+0000", 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043228+0000", 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043228+0000", 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6499999999999999e-07 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043231+0000", 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043237+0000", 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6630000000000004e-06 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043243+0000", 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4258 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043997+0000", 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013092919999999999, 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021250384000000001, 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4258 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043997+0000", 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043997+0000", 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043997+0000", 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044000+0000", 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.447e-06 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044007+0000", 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9770000000000003e-06 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044012+0000", 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4240000000000001e-06 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4291 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044702+0000", 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012387695000000001, 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020573530999999999, 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4291 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044702+0000", 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044702+0000", 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044702+0000", 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000002e-07 2024-09-17T23:30:48.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044705+0000", 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044711+0000", 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044716+0000", 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3149999999999999e-06 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4354 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046249+0000", 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010840326000000001, 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019057772000000001, 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4354 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046249+0000", 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046249+0000", 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046250+0000", 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1200000000000003e-07 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046252+0000", 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3130000000000001e-06 2024-09-17T23:30:48.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046258+0000", 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2929999999999997e-06 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046266+0000", 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2350000000000002e-06 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4452 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050703+0000", 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0063870300000000001, 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014632053000000001, 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4452 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050703+0000", 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050703+0000", 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050703+0000", 2024-09-17T23:30:48.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6699999999999999e-07 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050705+0000", 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.849e-06 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050711+0000", 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8019999999999997e-06 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050715+0000", 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4227 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043252+0000", 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013837838999999999, 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022111734000000001, 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4227 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043252+0000", 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043252+0000", 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043252+0000", 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-07 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043257+0000", 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043263+0000", 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.443e-06 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043268+0000", 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9949999999999996e-06 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4259 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044019+0000", 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013070231999999999, 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021371596, 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4259 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044019+0000", 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044019+0000", 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044020+0000", 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0699999999999998e-07 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044022+0000", 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.39e-06 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044029+0000", 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5939999999999999e-06 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044034+0000", 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0810000000000001e-06 2024-09-17T23:30:48.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4292 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044723+0000", 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012366754000000001, 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020707006, 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4292 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044723+0000", 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044723+0000", 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044723+0000", 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2399999999999999e-07 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044728+0000", 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9679999999999997e-06 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044734+0000", 2024-09-17T23:30:48.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1720000000000004e-06 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044739+0000", 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0350000000000001e-06 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4355 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046273+0000", 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010816762000000001, 2024-09-17T23:30:48.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019212354000000001, 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4355 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046273+0000", 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046273+0000", 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046273+0000", 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046276+0000", 2024-09-17T23:30:48.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3539999999999998e-06 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046282+0000", 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5849999999999996e-06 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046288+0000", 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4539999999999998e-06 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4397 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048268+0000", 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0088217629999999998, 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017278005999999999, 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4397 2024-09-17T23:30:48.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048268+0000", 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048268+0000", 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048269+0000", 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1459999999999999e-06 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048274+0000", 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.31e-06 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048287+0000", 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2826000000000001e-05 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048295+0000", 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2600000000000005e-06 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4453 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050721+0000", 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0063684730000000004, 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014885977, 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4453 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050721+0000", 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050721+0000", 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050722+0000", 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-07 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050724+0000", 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9209999999999999e-06 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050731+0000", 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.52e-06 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050736+0000", 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2650000000000004e-06 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4228 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043276+0000", 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013813784000000001, 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022387031000000002, 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4228 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043276+0000", 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043276+0000", 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043276+0000", 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3000000000000001e-07 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043278+0000", 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1440000000000001e-06 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043286+0000", 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5070000000000003e-06 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043291+0000", 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0950000000000003e-06 2024-09-17T23:30:48.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4260 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044052+0000", 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013037976, 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021640132999999999, 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4260 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044052+0000", 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044052+0000", 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044052+0000", 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044054+0000", 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.159e-06 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044062+0000", 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3429999999999998e-06 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044067+0000", 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4293 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044746+0000", 2024-09-17T23:30:48.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012343533, 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020975022999999999, 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4293 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044746+0000", 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044746+0000", 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044746+0000", 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7800000000000002e-07 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044749+0000", 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1459999999999999e-06 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044756+0000", 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1899999999999998e-06 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044761+0000", 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1850000000000003e-06 2024-09-17T23:30:48.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4356 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046295+0000", 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010794695, 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019454141000000001, 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4356 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046295+0000", 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046295+0000", 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046295+0000", 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4799999999999999e-07 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046297+0000", 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.069e-06 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046304+0000", 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4919999999999998e-06 2024-09-17T23:30:48.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046309+0000", 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0599999999999998e-06 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4399 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048309+0000", 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087805560000000001, 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017469074000000001, 2024-09-17T23:30:48.744 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4399 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048309+0000", 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048309+0000", 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048309+0000", 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048312+0000", 2024-09-17T23:30:48.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7640000000000001e-06 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048322+0000", 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3219999999999999e-06 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048329+0000", 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4454 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050749+0000", 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.006340147, 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015056406, 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4454 2024-09-17T23:30:48.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050749+0000", 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050749+0000", 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050750+0000", 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050752+0000", 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0269999999999998e-06 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050759+0000", 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050765+0000", 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1619999999999996e-06 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4229 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.043297+0000", 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013792095000000001, 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022535616000000001, 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4229 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043297+0000", 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043297+0000", 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043298+0000", 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043300+0000", 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043306+0000", 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2700000000000001e-06 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.043311+0000", 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0039999999999999e-06 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4261 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044073+0000", 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.013016049999999999, 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021789894000000001, 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4261 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044073+0000", 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044073+0000", 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044074+0000", 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044076+0000", 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6789999999999999e-06 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044081+0000", 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044086+0000", 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8590000000000004e-06 2024-09-17T23:30:48.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4294 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.044767+0000", 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.012322194999999999, 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021148442, 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4294 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044767+0000", 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044767+0000", 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044768+0000", 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044775+0000", 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.644e-06 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044782+0000", 2024-09-17T23:30:48.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8990000000000004e-06 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.044788+0000", 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4940000000000003e-06 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4357 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.046316+0000", 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010773886999999999, 2024-09-17T23:30:48.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019659929999999999, 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4357 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046316+0000", 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046316+0000", 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046316+0000", 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046326+0000", 2024-09-17T23:30:48.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0242e-05 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046333+0000", 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9059999999999996e-06 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.046339+0000", 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4960000000000004e-06 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4401 3.3 3.470a4023 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.048337+0000", 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0087529730000000007, 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01769451, 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4401 2024-09-17T23:30:48.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048337+0000", 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048337+0000", 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048337+0000", 2024-09-17T23:30:48.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1900000000000002e-07 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048339+0000", 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1569999999999998e-06 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048346+0000", 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6139999999999997e-06 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.048351+0000", 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5149999999999997e-06 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5306.0:4455 3.3 3.b9b36763 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:48.050773+0000", 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0063169350000000001, 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01531867, 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-17T23:30:48.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5306", 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/724879060", 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 4455 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050773+0000", 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:48.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050773+0000", 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050773+0000", 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.13e-07 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050780+0000", 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.357e-06 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:48.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050788+0000", 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7360000000000001e-06 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:48.050794+0000", 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4040000000000003e-06 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 195 2024-09-17T23:30:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:48.773 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T23:30:48.773 INFO:teuthology.orchestra.run.smithi139.stdout:4892 read 0x747e200 thru 0x7486bbb (0x89bc bytes) 2024-09-17T23:30:48.773 INFO:teuthology.orchestra.run.smithi139.stdout:4896 punch from 0x2e48000 to 0x2e54400, (0xc400 bytes) 2024-09-17T23:30:48.773 INFO:teuthology.orchestra.run.smithi139.stdout:4898 clone 33 order 23 su 131072 sc 11 2024-09-17T23:30:48.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:48.774 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:48.775 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:48.775 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:48.775 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:48.775 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:48.776 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:48.776 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:30:48.777 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:30:48.777 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:48.771+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:48.874 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:48.871+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:48.927 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:48.927 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:48.927 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:48.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:48.958 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:30:49.014 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:49.012+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:49.049 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone32 from 0x7921a00 (overlap 0x7921a00) to 0x0 2024-09-17T23:30:49.076 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:49.073+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:49.080 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:49.080 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.234983757999998, 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:30:49.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.225411658999999, 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:30:49.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:30:49.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.221881152999998, 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:49.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:30:49.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.220386446999999, 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:30:49.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.220203527999999, 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:49.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:30:49.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.220090027000001, 2024-09-17T23:30:49.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:30:49.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.22006592, 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:30:49.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.219991588999999, 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:30:49.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:30:49.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.219968040000001, 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:30:49.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.216796839000001, 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:30:49.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.212388503, 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:30:49.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:30:49.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.185116396000002, 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:30:49.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:30:49.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.175934244, 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.175902750999999, 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:30:49.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.175877059000001, 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:49.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.175732506999999, 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:49.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:30:49.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:30:49.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.174164390000001, 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:30:49.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:30:49.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.173967043000001, 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:30:49.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.173940728000002, 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:49.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.173912385000001, 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:30:49.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:30:49.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.184 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:49.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:49.186 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:49.184+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:49.287 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:49.285+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:49.335 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:49.336 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:49.336 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:49.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:49.389 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:49.387+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:49.451 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T23:30:49.455 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:49.455 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:49.456 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:49.456 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:49.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:49.488 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:49.486+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:49.507 INFO:teuthology.orchestra.run.smithi139.stdout:4899 write 0x6fbda00 thru 0x6fc6c7a (0x927b bytes) 2024-09-17T23:30:49.508 INFO:teuthology.orchestra.run.smithi139.stdout:4901 read 0x7199000 thru 0x719c8c3 (0x38c4 bytes) 2024-09-17T23:30:49.541 INFO:teuthology.orchestra.run.smithi139.stdout:4902 write 0xd433400 thru 0xd4395f2 (0x61f3 bytes) 2024-09-17T23:30:49.541 INFO:teuthology.orchestra.run.smithi139.stdout:4903 read 0x4ce8c00 thru 0x4ce8fa8 (0x3a9 bytes) 2024-09-17T23:30:49.550 INFO:teuthology.orchestra.run.smithi139.stdout:4907 trunc from 0xd439600 to 0x566a200 2024-09-17T23:30:49.591 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:49.587+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:49.641 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 110.167045024, 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.309209211000002, 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:49.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.308516918000002, 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:49.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:49.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.308505337, 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:49.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:49.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.307978089999999, 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:49.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.307944106999997, 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.307633514999999, 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:49.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.307607380999997, 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:49.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.307109474000001, 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:49.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.306508383999997, 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:49.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.305959749000003, 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:49.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.305752151, 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:49.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.296136250000004, 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:49.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:49.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.296126016000002, 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:49.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:49.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.296116148999999, 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:49.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:49.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.296106000999998, 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:49.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.295636133000002, 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:49.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.295614016999998, 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:49.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:49.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.295326322000001, 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:49.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.109940152, 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:49.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:49.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:49.744 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:49.746 INFO:teuthology.orchestra.run.smithi139.stdout:4908 trunc from 0x566a200 to 0x8ea5600 2024-09-17T23:30:49.746 INFO:teuthology.orchestra.run.smithi139.stdout:4910 read 0x6666200 thru 0x66685b3 (0x23b4 bytes) 2024-09-17T23:30:49.746 INFO:teuthology.orchestra.run.smithi139.stdout:4911 trunc from 0x8ea5600 to 0x3572200 2024-09-17T23:30:49.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:49.747 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:49.743+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:49.761 INFO:teuthology.orchestra.run.smithi139.stdout:4914 write 0x768a000 thru 0x768e855 (0x4856 bytes) 2024-09-17T23:30:49.762 INFO:teuthology.orchestra.run.smithi139.stdout:4915 write 0x6578400 thru 0x6581e21 (0x9a22 bytes) 2024-09-17T23:30:49.762 INFO:teuthology.orchestra.run.smithi139.stdout:4916 punch from 0x391ce00 to 0x3926800, (0x9a00 bytes) 2024-09-17T23:30:49.764 INFO:teuthology.orchestra.run.smithi139.stdout:4917 read 0x14ff600 thru 0x1507ce7 (0x86e8 bytes) 2024-09-17T23:30:49.767 INFO:teuthology.orchestra.run.smithi139.stdout:4918 write 0x369e000 thru 0x36a84c8 (0xa4c9 bytes) 2024-09-17T23:30:49.767 INFO:teuthology.orchestra.run.smithi139.stdout:4919 write 0x4eeda00 thru 0x4ef8305 (0xa906 bytes) 2024-09-17T23:30:49.768 INFO:teuthology.orchestra.run.smithi139.stdout:4921 write 0x342ea00 thru 0x3436a82 (0x8083 bytes) 2024-09-17T23:30:49.771 INFO:teuthology.orchestra.run.smithi139.stdout:4922 punch from 0x6ed600 to 0x6fb600, (0xe000 bytes) 2024-09-17T23:30:49.771 INFO:teuthology.orchestra.run.smithi139.stdout:4923 write 0x1bd9800 thru 0x1bddd90 (0x4591 bytes) 2024-09-17T23:30:49.772 INFO:teuthology.orchestra.run.smithi139.stdout:4925 read 0x6a51c00 thru 0x6a5568b (0x3a8c bytes) 2024-09-17T23:30:49.772 INFO:teuthology.orchestra.run.smithi139.stdout:4926 flatten 2024-09-17T23:30:49.785 INFO:teuthology.orchestra.run.smithi139.stdout:4927 read 0x23bb600 thru 0x23c9f4c (0xe94d bytes) 2024-09-17T23:30:49.788 INFO:teuthology.orchestra.run.smithi139.stdout:4928 trunc from 0x768ea00 to 0x93e0400 2024-09-17T23:30:49.804 INFO:teuthology.orchestra.run.smithi139.stdout:4929 read 0x4a7c400 thru 0x4a897f5 (0xd3f6 bytes) 2024-09-17T23:30:49.810 INFO:teuthology.orchestra.run.smithi139.stdout:4930 trunc from 0x93e0400 to 0xe2ef200 2024-09-17T23:30:49.825 INFO:teuthology.orchestra.run.smithi139.stdout:4931 write 0x1f7f000 thru 0x1f89c9f (0xaca0 bytes) 2024-09-17T23:30:49.825 INFO:teuthology.orchestra.run.smithi139.stdout:4933 write 0xc77e400 thru 0xc77f27a (0xe7b bytes) 2024-09-17T23:30:49.825 INFO:teuthology.orchestra.run.smithi139.stdout:4937 read 0xc51ba00 thru 0xc526110 (0xa711 bytes) 2024-09-17T23:30:49.825 INFO:teuthology.orchestra.run.smithi139.stdout:4940 punch from 0x51f8000 to 0x5202c00, (0xac00 bytes) 2024-09-17T23:30:49.826 INFO:teuthology.orchestra.run.smithi139.stdout:4942 read 0xa663200 thru 0xa666bee (0x39ef bytes) 2024-09-17T23:30:49.826 INFO:teuthology.orchestra.run.smithi139.stdout:4943 read 0x4785a00 thru 0x4791fcd (0xc5ce bytes) 2024-09-17T23:30:49.826 INFO:teuthology.orchestra.run.smithi139.stdout:4946 punch from 0xbb4f400 to 0xbb52c00, (0x3800 bytes) 2024-09-17T23:30:49.826 INFO:teuthology.orchestra.run.smithi139.stdout:4948 write 0x3efda00 thru 0x3f0c743 (0xed44 bytes) 2024-09-17T23:30:49.830 INFO:teuthology.orchestra.run.smithi139.stdout:4949 trunc from 0xe2ef200 to 0xd016c00 2024-09-17T23:30:49.845 INFO:teuthology.orchestra.run.smithi139.stdout:4951 write 0xc676000 thru 0xc6766bf (0x6c0 bytes) 2024-09-17T23:30:49.846 INFO:teuthology.orchestra.run.smithi139.stdout:4954 read 0xac82000 thru 0xac82b6e (0xb6f bytes) 2024-09-17T23:30:49.846 INFO:teuthology.orchestra.run.smithi139.stdout:4955 read 0x7209c00 thru 0x720cc0b (0x300c bytes) 2024-09-17T23:30:49.846 INFO:teuthology.orchestra.run.smithi139.stdout:4956 read 0x4203600 thru 0x42042ff (0xd00 bytes) 2024-09-17T23:30:49.847 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:49.844+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:49.847 INFO:teuthology.orchestra.run.smithi139.stdout:4959 read 0x1a15000 thru 0x1a1f8e2 (0xa8e3 bytes) 2024-09-17T23:30:49.848 INFO:teuthology.orchestra.run.smithi139.stdout:4960 read 0x88ab800 thru 0x88baae8 (0xf2e9 bytes) 2024-09-17T23:30:49.856 INFO:teuthology.orchestra.run.smithi139.stdout:4961 trunc from 0xd016c00 to 0x59a7e00 2024-09-17T23:30:49.878 INFO:teuthology.orchestra.run.smithi139.stdout:4964 read 0x4d99c00 thru 0x4d9da7e (0x3e7f bytes) 2024-09-17T23:30:49.894 INFO:teuthology.orchestra.run.smithi139.stdout:4968 write 0x99b4e00 thru 0x99ba3f3 (0x55f4 bytes) 2024-09-17T23:30:49.898 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:49.898 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:49.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:49.911 INFO:teuthology.orchestra.run.smithi139.stdout:4971 write 0xdc85600 thru 0xdc85726 (0x127 bytes) 2024-09-17T23:30:49.918 INFO:teuthology.orchestra.run.smithi139.stdout:4972 write 0xa769600 thru 0xa778c7a (0xf67b bytes) 2024-09-17T23:30:49.918 INFO:teuthology.orchestra.run.smithi139.stdout:4973 read 0x1ea7800 thru 0x1eb52f4 (0xdaf5 bytes) 2024-09-17T23:30:49.919 INFO:teuthology.orchestra.run.smithi139.stdout:4974 read 0x4908c00 thru 0x490cb13 (0x3f14 bytes) 2024-09-17T23:30:49.919 INFO:teuthology.orchestra.run.smithi139.stdout:4975 read 0x3dbac00 thru 0x3dbb998 (0xd99 bytes) 2024-09-17T23:30:49.933 INFO:teuthology.orchestra.run.smithi139.stdout:4976 write 0xe04ca00 thru 0xe0561f0 (0x97f1 bytes) 2024-09-17T23:30:49.941 INFO:teuthology.orchestra.run.smithi139.stdout:4977 write 0xa30c800 thru 0xa31570b (0x8f0c bytes) 2024-09-17T23:30:49.941 INFO:teuthology.orchestra.run.smithi139.stdout:4980 write 0xdadc000 thru 0xdae6aa5 (0xaaa6 bytes) 2024-09-17T23:30:49.947 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:30:49.945+0000 7f88010de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32005) UID: 0 2024-09-17T23:30:49.949 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:49.949 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:30:49.950 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-17T23:30:49.950 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T23:30:49.950 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:30:49.988 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T23:30:49.988 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-17T23:30:49.988 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:30:49.988 DEBUG:teuthology.orchestra.run.smithi037:> type ceph-objectstore-tool 2024-09-17T23:30:50.043 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T23:30:50.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-17T23:30:50.055 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:50.056 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:50.056 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:50.056 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:50.056 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:50.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:50.084 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:50.082+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:50.149 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:50.146+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:50.172 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.133824801000003, 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.133651861000004, 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.133611990999995, 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.839107764000005, 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.591406215000006, 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.581473948999999, 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.568246899999998, 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027717744, 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027696354, 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027679225, 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027661942999998, 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027644914, 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027612040000001, 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.027024333999996, 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.005232192999998, 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.004695005000002, 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.004208888999997, 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.932066286999998, 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.932044476000002, 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.922601606000001, 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:50.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:50.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:50.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:50.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:50.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:50.278 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:50.279 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:50.276+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:50.379 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:50.377+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:50.386 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:50.387 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:50.387 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:50.387 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:50.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:50.480 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:50.478+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:50.503 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:50.503 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:50.503 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:50.503 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:50.503 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:50.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:50.596 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:50.594+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:50.661 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.783940385999998, 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:50.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:50.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:50.668 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768315255999994, 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:50.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768277600000005, 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.674 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:50.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768260169000001, 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:50.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768225237999999, 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:50.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:50.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768193818, 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:50.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:50.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768177191999996, 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:50.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:50.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:50.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768155497999999, 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.701 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 83.768143780000003, 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:50.706 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:50.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:50.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.809005260999999, 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.712 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:30:50.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:30:50.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:30:50.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:50.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:50.719 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.805050096999999, 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:30:50.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:30:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:30:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.805009438999999, 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:30:50.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:30:50.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.804852703999998, 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:50.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:30:50.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.804750014, 2024-09-17T23:30:50.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:30:50.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.742 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.804713792000001, 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:50.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.747 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.804681497000001, 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:30:50.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:30:50.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 29.804526685999999, 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:30:50.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:50.758 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.804240365999998, 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:30:50.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:30:50.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.804171705000002, 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:30:50.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:30:50.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:30:50.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.804129731, 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:50.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:50.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:30:50.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:30:50.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:50.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:50.774 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:50.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:50.776 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:50.773+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:50.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:30:50.877 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:50.875+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:50.909 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:50.909 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:50.926 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:50.930 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:50.930 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:50.930 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:50.930 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:50.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:50.942 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:50.942 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:50.960 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:50.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:30:50.977 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:50.978 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:50.978 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:50.975+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:51.079 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:51.076+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:51.087 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.087 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:51.180 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:51.178+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:51.202 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.202 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:51.281 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:51.278+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:51.318 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.319 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:51.381 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:51.380+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:51.475 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.475 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.482 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:51.481+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:51.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:51.583 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:51.581+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.591 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.a 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.1e 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.1f 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.1c 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.17 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.11 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:3.0 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:2.1 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:2.2 2024-09-17T23:30:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:2.0 2024-09-17T23:30:51.622 INFO:teuthology.orchestra.run.smithi037.stdout:2.4 2024-09-17T23:30:51.622 INFO:teuthology.orchestra.run.smithi037.stdout:1.0 2024-09-17T23:30:51.684 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:51.681+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:51.747 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.748 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:51.785 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:51.783+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:51.886 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:51.884+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:51.904 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:51.904 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:51.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:30:51.974 INFO:teuthology.orchestra.run.smithi139.stdout:4981 write 0xe251600 thru 0xe252ce9 (0x16ea bytes) 2024-09-17T23:30:51.975 INFO:teuthology.orchestra.run.smithi139.stdout:4983 write 0x2cc5c00 thru 0x2ccbc44 (0x6045 bytes) 2024-09-17T23:30:51.982 INFO:teuthology.orchestra.run.smithi139.stdout:4984 trunc from 0xe252e00 to 0xc6bfc00 2024-09-17T23:30:51.986 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:51.984+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:52.000 INFO:teuthology.orchestra.run.smithi139.stdout:4986 read 0x9d43c00 thru 0x9d4c281 (0x8682 bytes) 2024-09-17T23:30:52.000 INFO:teuthology.orchestra.run.smithi139.stdout:4987 read 0x749aa00 thru 0x74a8f10 (0xe511 bytes) 2024-09-17T23:30:52.001 INFO:teuthology.orchestra.run.smithi139.stdout:4988 read 0xbc1f600 thru 0xbc21118 (0x1b19 bytes) 2024-09-17T23:30:52.001 INFO:teuthology.orchestra.run.smithi139.stdout:4989 read 0xaf61200 thru 0xaf61e27 (0xc28 bytes) 2024-09-17T23:30:52.001 INFO:teuthology.orchestra.run.smithi139.stdout:4990 punch from 0xf97e00 to 0xf9d600, (0x5800 bytes) 2024-09-17T23:30:52.002 INFO:teuthology.orchestra.run.smithi139.stdout:4993 read 0x7eb4200 thru 0x7ebfdca (0xbbcb bytes) 2024-09-17T23:30:52.004 INFO:teuthology.orchestra.run.smithi139.stdout:4996 trunc from 0xc6bfc00 to 0xb42a600 2024-09-17T23:30:52.019 INFO:teuthology.orchestra.run.smithi139.stdout:4997 write 0x3b1b200 thru 0x3b2612a (0xaf2b bytes) 2024-09-17T23:30:52.019 INFO:teuthology.orchestra.run.smithi139.stdout:4998 write 0x416e00 thru 0x420cc3 (0x9ec4 bytes) 2024-09-17T23:30:52.020 INFO:teuthology.orchestra.run.smithi139.stdout:5000 punch from 0xaa56000 to 0xaa62800, (0xc800 bytes) 2024-09-17T23:30:52.021 INFO:teuthology.orchestra.run.smithi139.stdout:5001 write 0x67f5e00 thru 0x67f62d1 (0x4d2 bytes) 2024-09-17T23:30:52.022 INFO:teuthology.orchestra.run.smithi139.stdout:5003 write 0x516000 thru 0x51d783 (0x7784 bytes) 2024-09-17T23:30:52.028 INFO:teuthology.orchestra.run.smithi139.stdout:5004 trunc from 0xb42a600 to 0xd3bc600 2024-09-17T23:30:52.029 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.030 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:52.045 INFO:teuthology.orchestra.run.smithi139.stdout:5005 clone 34 order 23 su 65536 sc 14 2024-09-17T23:30:52.087 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:52.085+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:52.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 2.1 --file /var/log/ceph/exp.2.1.5 2024-09-17T23:30:52.186 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.186 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:52.225 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:52.223+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:52.289 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:52.287+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:52.301 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.302 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:52.390 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:52.388+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:52.462 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.462 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:52.491 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:52.488+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:52.581 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.581 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.590 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:52.591 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:52.589+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:52.692 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:52.690+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:52.695 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.696 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:52.793 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:52.791+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:52.855 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.855 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:52.910 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:52.908+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:52.977 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:52.978 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:52.988 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:52.994 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:52.991+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:53.095 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:53.093+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:53.136 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:53.136 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:53.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:53.196 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:53.194+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:53.297 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:53.297 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:53.297 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:53.295+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:53.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:53.398 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:53.395+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:53.403 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T23:30:53.426 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:53.426 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:53.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:53.460 INFO:teuthology.orchestra.run.smithi139.stdout:5006 read 0x690aa00 thru 0x690b0d6 (0x6d7 bytes) 2024-09-17T23:30:53.461 INFO:teuthology.orchestra.run.smithi139.stdout:5007 write 0x83db800 thru 0x83eaa27 (0xf228 bytes) 2024-09-17T23:30:53.471 INFO:teuthology.orchestra.run.smithi139.stdout:5009 trunc from 0xd3bc600 to 0x60a8c00 2024-09-17T23:30:53.520 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:53.517+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:53.600 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:53.598+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:53.601 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:53.601 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:53.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:53.614 INFO:teuthology.orchestra.run.smithi037.stderr:Exporting 2.1 info 2.1( empty local-lis/les=15/16 n=0 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) 2024-09-17T23:30:53.618 INFO:teuthology.orchestra.run.smithi139.stdout:5011 read 0x2acee00 thru 0x2ad60d4 (0x72d5 bytes) 2024-09-17T23:30:53.620 INFO:teuthology.orchestra.run.smithi139.stdout:5012 read 0x1a0be00 thru 0x1a0f9b6 (0x3bb7 bytes) 2024-09-17T23:30:53.620 INFO:teuthology.orchestra.run.smithi139.stdout:5014 write 0x1906a00 thru 0x190f740 (0x8d41 bytes) 2024-09-17T23:30:53.620 INFO:teuthology.orchestra.run.smithi037.stderr:Export successful 2024-09-17T23:30:53.621 INFO:teuthology.orchestra.run.smithi139.stdout:5019 trunc from 0x60a8c00 to 0x5791e00 2024-09-17T23:30:53.644 INFO:teuthology.orchestra.run.smithi139.stdout:5021 write 0x105c400 thru 0x106a215 (0xde16 bytes) 2024-09-17T23:30:53.645 INFO:teuthology.orchestra.run.smithi139.stdout:5022 read 0x37ef400 thru 0x37f52a3 (0x5ea4 bytes) 2024-09-17T23:30:53.645 INFO:teuthology.orchestra.run.smithi139.stdout:5023 write 0x4b5ca00 thru 0x4b63b3d (0x713e bytes) 2024-09-17T23:30:53.645 INFO:teuthology.orchestra.run.smithi139.stdout:5026 read 0x320da00 thru 0x3214bd2 (0x71d3 bytes) 2024-09-17T23:30:53.665 INFO:teuthology.orchestra.run.smithi139.stdout:5027 write 0x9c18600 thru 0x9c252c4 (0xccc5 bytes) 2024-09-17T23:30:53.676 INFO:teuthology.orchestra.run.smithi139.stdout:5029 trunc from 0x9c25400 to 0x8c6a00 2024-09-17T23:30:53.701 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:53.698+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:53.733 INFO:teuthology.orchestra.run.smithi139.stdout:5035 write 0x5cc4e00 thru 0x5cccae9 (0x7cea bytes) 2024-09-17T23:30:53.739 INFO:teuthology.orchestra.run.smithi139.stdout:5036 flatten 2024-09-17T23:30:53.750 INFO:teuthology.orchestra.run.smithi139.stdout:5038 read 0x3ac6800 thru 0x3acff64 (0x9765 bytes) 2024-09-17T23:30:53.751 INFO:teuthology.orchestra.run.smithi139.stdout:5039 read 0x2d7b000 thru 0x2d8acc7 (0xfcc8 bytes) 2024-09-17T23:30:53.762 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:53.762 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:53.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:53.774 INFO:teuthology.orchestra.run.smithi139.stdout:5041 write 0xe8ed200 thru 0xe8f7673 (0xa474 bytes) 2024-09-17T23:30:53.775 INFO:teuthology.orchestra.run.smithi139.stdout:5042 read 0x9952a00 thru 0x9960295 (0xd896 bytes) 2024-09-17T23:30:53.775 INFO:teuthology.orchestra.run.smithi139.stdout:5044 write 0x394e200 thru 0x395cd29 (0xeb2a bytes) 2024-09-17T23:30:53.780 INFO:teuthology.orchestra.run.smithi139.stdout:5046 read 0xd3f8000 thru 0xd405e03 (0xde04 bytes) 2024-09-17T23:30:53.780 INFO:teuthology.orchestra.run.smithi139.stdout:5049 read 0x408600 thru 0x408aa0 (0x4a1 bytes) 2024-09-17T23:30:53.781 INFO:teuthology.orchestra.run.smithi139.stdout:5051 write 0x21e6600 thru 0x21f5dc1 (0xf7c2 bytes) 2024-09-17T23:30:53.787 INFO:teuthology.orchestra.run.smithi139.stdout:5053 trunc from 0xe8f7800 to 0x91ff400 2024-09-17T23:30:53.814 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:53.811+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:53.833 INFO:teuthology.orchestra.run.smithi139.stdout:5055 write 0xe1f4400 thru 0xe1fb748 (0x7349 bytes) 2024-09-17T23:30:53.834 INFO:teuthology.orchestra.run.smithi139.stdout:5057 read 0x998ba00 thru 0x9998f17 (0xd518 bytes) 2024-09-17T23:30:53.834 INFO:teuthology.orchestra.run.smithi139.stdout:5059 write 0x482800 thru 0x4846c3 (0x1ec4 bytes) 2024-09-17T23:30:53.835 INFO:teuthology.orchestra.run.smithi139.stdout:5060 write 0x6c46400 thru 0x6c4b726 (0x5327 bytes) 2024-09-17T23:30:53.835 INFO:teuthology.orchestra.run.smithi139.stdout:5062 punch from 0xd675800 to 0xd678a00, (0x3200 bytes) 2024-09-17T23:30:53.844 INFO:teuthology.orchestra.run.smithi139.stdout:5065 trunc from 0xe1fb800 to 0x6f9e200 2024-09-17T23:30:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:5067 trunc from 0x6f9e200 to 0x505fc00 2024-09-17T23:30:53.888 INFO:teuthology.orchestra.run.smithi139.stdout:5068 read 0xe39c00 thru 0xe41a72 (0x7e73 bytes) 2024-09-17T23:30:53.892 INFO:teuthology.orchestra.run.smithi139.stdout:5069 trunc from 0x505fc00 to 0x2a9c800 2024-09-17T23:30:53.902 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:53.900+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:53.909 INFO:teuthology.orchestra.run.smithi139.stdout:5072 punch from 0xed7c00 to 0xedde00, (0x6200 bytes) 2024-09-17T23:30:53.922 INFO:teuthology.orchestra.run.smithi139.stdout:5075 write 0x4229200 thru 0x42331f3 (0x9ff4 bytes) 2024-09-17T23:30:53.926 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:53.926 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:53.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:53.949 INFO:teuthology.orchestra.run.smithi139.stdout:5076 write 0xb897000 thru 0xb8a3d11 (0xcd12 bytes) 2024-09-17T23:30:53.950 INFO:teuthology.orchestra.run.smithi139.stdout:5080 read 0x1ed0c00 thru 0x1ed9ba0 (0x8fa1 bytes) 2024-09-17T23:30:53.968 INFO:teuthology.orchestra.run.smithi139.stdout:5081 write 0xd15a000 thru 0xd15c285 (0x2286 bytes) 2024-09-17T23:30:53.968 INFO:teuthology.orchestra.run.smithi139.stdout:5082 write 0xd73800 thru 0xd787e8 (0x4fe9 bytes) 2024-09-17T23:30:53.969 INFO:teuthology.orchestra.run.smithi139.stdout:5085 write 0x213aa00 thru 0x21478cc (0xcecd bytes) 2024-09-17T23:30:53.970 INFO:teuthology.orchestra.run.smithi139.stdout:5086 write 0xca62800 thru 0xca6d4c1 (0xacc2 bytes) 2024-09-17T23:30:53.974 INFO:teuthology.orchestra.run.smithi139.stdout:5087 trunc from 0xd15c400 to 0x99f0c00 2024-09-17T23:30:53.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:30:53.997 INFO:teuthology.orchestra.run.smithi139.stdout:5089 punch from 0x6e3a600 to 0x6e3f000, (0x4a00 bytes) 2024-09-17T23:30:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:5092 read 0x5336000 thru 0x533fe6e (0x9e6f bytes) 2024-09-17T23:30:54.002 INFO:teuthology.orchestra.run.smithi139.stdout:5095 write 0x5bc5600 thru 0x5bc8cc7 (0x36c8 bytes) 2024-09-17T23:30:54.003 INFO:teuthology.orchestra.run.smithi139.stdout:5097 read 0x82bae00 thru 0x82c6d5e (0xbf5f bytes) 2024-09-17T23:30:54.003 INFO:teuthology.orchestra.run.smithi139.stdout:5098 write 0x3adfc00 thru 0x3adfed6 (0x2d7 bytes) 2024-09-17T23:30:54.003 INFO:teuthology.orchestra.run.smithi139.stdout:5102 read 0x2eb4200 thru 0x2ebf0f6 (0xaef7 bytes) 2024-09-17T23:30:54.019 INFO:teuthology.orchestra.run.smithi139.stdout:5103 write 0xb777400 thru 0xb78583e (0xe43f bytes) 2024-09-17T23:30:54.020 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:54.017+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:54.020 INFO:teuthology.orchestra.run.smithi139.stdout:5105 read 0x2a3a000 thru 0x2a410e6 (0x70e7 bytes) 2024-09-17T23:30:54.021 INFO:teuthology.orchestra.run.smithi139.stdout:5107 trunc from 0xb785a00 to 0xb0f5200 2024-09-17T23:30:54.046 INFO:teuthology.orchestra.run.smithi139.stdout:5110 punch from 0x9f3da00 to 0x9f47400, (0x9a00 bytes) 2024-09-17T23:30:54.048 INFO:teuthology.orchestra.run.smithi139.stdout:5112 read 0x830fa00 thru 0x83149c9 (0x4fca bytes) 2024-09-17T23:30:54.051 INFO:teuthology.orchestra.run.smithi139.stdout:5113 trunc from 0xb0f5200 to 0xca45200 2024-09-17T23:30:54.061 INFO:teuthology.orchestra.run.smithi139.stdout:5115 read 0x1e62200 thru 0x1e65a13 (0x3814 bytes) 2024-09-17T23:30:54.067 INFO:teuthology.orchestra.run.smithi139.stdout:5118 trunc from 0xca45200 to 0xa87e600 2024-09-17T23:30:54.087 INFO:teuthology.orchestra.run.smithi139.stdout:5120 trunc from 0xa87e600 to 0x92db000 2024-09-17T23:30:54.089 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.089 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:54.108 INFO:teuthology.orchestra.run.smithi139.stdout:5121 read 0x7c46800 thru 0x7c4b615 (0x4e16 bytes) 2024-09-17T23:30:54.109 INFO:teuthology.orchestra.run.smithi139.stdout:5123 punch from 0x78dc200 to 0x78dfe00, (0x3c00 bytes) 2024-09-17T23:30:54.129 INFO:teuthology.orchestra.run.smithi139.stdout:5125 write 0xaa70600 thru 0xaa71394 (0xd95 bytes) 2024-09-17T23:30:54.136 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 2.1 2024-09-17T23:30:54.144 INFO:teuthology.orchestra.run.smithi139.stdout:5129 write 0xbb09000 thru 0xbb100d2 (0x70d3 bytes) 2024-09-17T23:30:54.145 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:54.142+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:54.146 INFO:teuthology.orchestra.run.smithi139.stdout:5131 punch from 0xb9d200 to 0xba5000, (0x7e00 bytes) 2024-09-17T23:30:54.147 INFO:teuthology.orchestra.run.smithi139.stdout:5132 read 0x3cf4600 thru 0x3cfd390 (0x8d91 bytes) 2024-09-17T23:30:54.147 INFO:teuthology.orchestra.run.smithi139.stdout:5136 read 0x977be00 thru 0x977c6a1 (0x8a2 bytes) 2024-09-17T23:30:54.148 INFO:teuthology.orchestra.run.smithi139.stdout:5137 read 0x9f3c00 thru 0x9f6055 (0x2456 bytes) 2024-09-17T23:30:54.148 INFO:teuthology.orchestra.run.smithi139.stdout:5139 read 0x94d0200 thru 0x94d3e0c (0x3c0d bytes) 2024-09-17T23:30:54.149 INFO:teuthology.orchestra.run.smithi139.stdout:5141 read 0x63f0a00 thru 0x63f93f5 (0x89f6 bytes) 2024-09-17T23:30:54.153 INFO:teuthology.orchestra.run.smithi139.stdout:5144 trunc from 0xbb10200 to 0x7fb0400 2024-09-17T23:30:54.174 INFO:teuthology.orchestra.run.smithi139.stdout:5145 trunc from 0x7fb0400 to 0x1daae00 2024-09-17T23:30:54.204 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:54.202+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:54.210 INFO:teuthology.orchestra.run.smithi139.stdout:5147 write 0x2d54800 thru 0x2d5e2bf (0x9ac0 bytes) 2024-09-17T23:30:54.218 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.218 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.227 INFO:teuthology.orchestra.run.smithi139.stdout:5149 write 0x4322e00 thru 0x432d8e9 (0xaaea bytes) 2024-09-17T23:30:54.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:54.255 INFO:teuthology.orchestra.run.smithi139.stdout:5151 write 0xe2da600 thru 0xe2e0398 (0x5d99 bytes) 2024-09-17T23:30:54.255 INFO:teuthology.orchestra.run.smithi139.stdout:5152 read 0x7986a00 thru 0x7990fb2 (0xa5b3 bytes) 2024-09-17T23:30:54.256 INFO:teuthology.orchestra.run.smithi139.stdout:5153 punch from 0xb99dc00 to 0xb9a0c00, (0x3000 bytes) 2024-09-17T23:30:54.257 INFO:teuthology.orchestra.run.smithi139.stdout:5154 read 0x8a03800 thru 0x8a077cd (0x3fce bytes) 2024-09-17T23:30:54.257 INFO:teuthology.orchestra.run.smithi139.stdout:5157 write 0x6b9f000 thru 0x6b9f44f (0x450 bytes) 2024-09-17T23:30:54.260 INFO:teuthology.orchestra.run.smithi139.stdout:5159 write 0x7aea600 thru 0x7af4262 (0x9c63 bytes) 2024-09-17T23:30:54.266 INFO:teuthology.orchestra.run.smithi139.stdout:5160 trunc from 0xe2e0400 to 0x8f2de00 2024-09-17T23:30:54.309 INFO:teuthology.orchestra.run.smithi139.stdout:5166 write 0x9706c00 thru 0x971199a (0xad9b bytes) 2024-09-17T23:30:54.310 INFO:teuthology.orchestra.run.smithi139.stdout:5170 read 0xa92e00 thru 0xa9fdc8 (0xcfc9 bytes) 2024-09-17T23:30:54.310 INFO:teuthology.orchestra.run.smithi139.stdout:5172 write 0x5abc800 thru 0x5acc721 (0xff22 bytes) 2024-09-17T23:30:54.310 INFO:teuthology.orchestra.run.smithi139.stdout:5175 read 0x4840e00 thru 0x4840e4a (0x4b bytes) 2024-09-17T23:30:54.312 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:54.310+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:54.320 INFO:teuthology.orchestra.run.smithi139.stdout:5177 trunc from 0x9711a00 to 0x1598000 2024-09-17T23:30:54.373 INFO:teuthology.orchestra.run.smithi139.stdout:5178 punch from 0x1206000 to 0x1210400, (0xa400 bytes) 2024-09-17T23:30:54.382 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.382 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:54.400 INFO:teuthology.orchestra.run.smithi139.stdout:5181 write 0x9f4ba00 thru 0x9f5b65e (0xfc5f bytes) 2024-09-17T23:30:54.400 INFO:teuthology.orchestra.run.smithi139.stdout:5182 write 0x5a4ba00 thru 0x5a506d8 (0x4cd9 bytes) 2024-09-17T23:30:54.400 INFO:teuthology.orchestra.run.smithi139.stdout:5183 read 0x22b3a00 thru 0x22b6515 (0x2b16 bytes) 2024-09-17T23:30:54.401 INFO:teuthology.orchestra.run.smithi139.stdout:5184 write 0x9cc9000 thru 0x9ccc612 (0x3613 bytes) 2024-09-17T23:30:54.404 INFO:teuthology.orchestra.run.smithi139.stdout:5185 trunc from 0x9f5b800 to 0x6f17000 2024-09-17T23:30:54.434 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:54.431+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:54.448 INFO:teuthology.orchestra.run.smithi139.stdout:5189 write 0x764f000 thru 0x7655efb (0x6efc bytes) 2024-09-17T23:30:54.455 INFO:teuthology.orchestra.run.smithi139.stdout:5193 punch from 0x3dcc00 to 0x3df000, (0x2400 bytes) 2024-09-17T23:30:54.457 INFO:teuthology.orchestra.run.smithi139.stdout:5194 write 0x546f600 thru 0x54771a7 (0x7ba8 bytes) 2024-09-17T23:30:54.475 INFO:teuthology.orchestra.run.smithi139.stdout:5195 write 0x8e13600 thru 0x8e205fc (0xcffd bytes) 2024-09-17T23:30:54.475 INFO:teuthology.orchestra.run.smithi139.stdout:5196 read 0x743e600 thru 0x743f914 (0x1315 bytes) 2024-09-17T23:30:54.475 INFO:teuthology.orchestra.run.smithi139.stdout:5197 read 0x623200 thru 0x626eb7 (0x3cb8 bytes) 2024-09-17T23:30:54.476 INFO:teuthology.orchestra.run.smithi139.stdout:5198 write 0x5b10400 thru 0x5b11e80 (0x1a81 bytes) 2024-09-17T23:30:54.476 INFO:teuthology.orchestra.run.smithi139.stdout:5199 read 0x2a5f200 thru 0x2a6e6d6 (0xf4d7 bytes) 2024-09-17T23:30:54.476 INFO:teuthology.orchestra.run.smithi139.stdout:5202 read 0x6d0aa00 thru 0x6d183a5 (0xd9a6 bytes) 2024-09-17T23:30:54.476 INFO:teuthology.orchestra.run.smithi139.stdout:5203 punch from 0x5dffc00 to 0x5e0d200, (0xd600 bytes) 2024-09-17T23:30:54.493 INFO:teuthology.orchestra.run.smithi139.stdout:5204 write 0xd8f7400 thru 0xd8f8a0f (0x1610 bytes) 2024-09-17T23:30:54.494 INFO:teuthology.orchestra.run.smithi139.stdout:5206 write 0xba1c200 thru 0xba2459b (0x839c bytes) 2024-09-17T23:30:54.494 INFO:teuthology.orchestra.run.smithi139.stdout:5208 read 0x7c5d400 thru 0x7c5eefb (0x1afc bytes) 2024-09-17T23:30:54.506 INFO:teuthology.orchestra.run.smithi139.stdout:5209 trunc from 0xd8f8c00 to 0x3aa6400 2024-09-17T23:30:54.507 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:54.504+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:54.544 INFO:teuthology.orchestra.run.smithi139.stdout:5210 trunc from 0x3aa6400 to 0xb450a00 2024-09-17T23:30:54.549 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.549 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.553 INFO:teuthology.orchestra.run.smithi139.stdout:5211 punch from 0x83b400 to 0x842800, (0x7400 bytes) 2024-09-17T23:30:54.555 INFO:teuthology.orchestra.run.smithi139.stdout:5212 punch from 0x75b1400 to 0x75b9000, (0x7c00 bytes) 2024-09-17T23:30:54.556 INFO:teuthology.orchestra.run.smithi139.stdout:5213 read 0xfa4200 thru 0xfa7e16 (0x3c17 bytes) 2024-09-17T23:30:54.556 INFO:teuthology.orchestra.run.smithi139.stdout:5214 punch from 0xab88e00 to 0xab96a00, (0xdc00 bytes) 2024-09-17T23:30:54.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:54.560 INFO:teuthology.orchestra.run.smithi139.stdout:5216 trunc from 0xb450a00 to 0xe6d3a00 2024-09-17T23:30:54.588 INFO:teuthology.orchestra.run.smithi139.stdout:5217 trunc from 0xe6d3a00 to 0x141f400 2024-09-17T23:30:54.608 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:54.606+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:54.627 INFO:teuthology.orchestra.run.smithi139.stdout:5222 trunc from 0x141f400 to 0x37cc400 2024-09-17T23:30:54.636 INFO:teuthology.orchestra.run.smithi139.stdout:5223 punch from 0x941400 to 0x948600, (0x7200 bytes) 2024-09-17T23:30:54.638 INFO:teuthology.orchestra.run.smithi139.stdout:5224 write 0xb76e00 thru 0xb81487 (0xa688 bytes) 2024-09-17T23:30:54.639 INFO:teuthology.orchestra.run.smithi139.stdout:5226 read 0x3bd600 thru 0x3cbece (0xe8cf bytes) 2024-09-17T23:30:54.639 INFO:teuthology.orchestra.run.smithi139.stdout:5227 punch from 0x9d9000 to 0x9df800, (0x6800 bytes) 2024-09-17T23:30:54.640 INFO:teuthology.orchestra.run.smithi139.stdout:5228 punch from 0x846a00 to 0x850400, (0x9a00 bytes) 2024-09-17T23:30:54.640 INFO:teuthology.orchestra.run.smithi139.stdout:5229 punch from 0x36fb200 to 0x36fd800, (0x2600 bytes) 2024-09-17T23:30:54.640 INFO:teuthology.orchestra.run.smithi139.stdout:5230 read 0x2aca600 thru 0x2acccde (0x26df bytes) 2024-09-17T23:30:54.640 INFO:teuthology.orchestra.run.smithi139.stdout:5231 write 0x1a1cc00 thru 0x1a2aa2b (0xde2c bytes) 2024-09-17T23:30:54.641 INFO:teuthology.orchestra.run.smithi139.stdout:5233 read 0x2d98800 thru 0x2d999d4 (0x11d5 bytes) 2024-09-17T23:30:54.642 INFO:teuthology.orchestra.run.smithi139.stdout:5234 punch from 0x7e2400 to 0x7e7400, (0x5000 bytes) 2024-09-17T23:30:54.661 INFO:teuthology.orchestra.run.smithi139.stdout:5235 write 0xb5c7c00 thru 0xb5cd8ef (0x5cf0 bytes) 2024-09-17T23:30:54.662 INFO:teuthology.orchestra.run.smithi139.stdout:5238 clone 35 order 18 su 2048 sc 11 2024-09-17T23:30:54.709 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:54.706+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:54.711 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.712 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.720 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:54.810 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:54.808+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:54.828 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.828 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:54.912 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:54.909+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:54.944 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:54.944 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:54.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:54.976 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:30:55.032 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:55.029+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:55.080 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.080 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:55.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:55.136 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:55.134+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:55.203 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.203 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:55.212 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:55.253 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:55.251+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:55.315 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:55.313+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:55.369 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.369 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:55.378 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:55.393 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T23:30:55.415 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:55.413+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:55.516 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:55.514+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:55.543 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.543 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:55.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:55.617 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:55.614+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:55.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:55.717 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:55.716+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:55.819 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:55.815+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:55.835 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.835 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:55.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:55.891 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:30:55.894 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:30:55.895 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:30:55.920 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:55.918+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:55.993 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:55.993 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:56.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:56.038 INFO:teuthology.orchestra.run.smithi139.stdout:5239 read 0x968a800 thru 0x9696e81 (0xc682 bytes) 2024-09-17T23:30:56.039 INFO:teuthology.orchestra.run.smithi139.stdout:5241 read 0x7434600 thru 0x74418fb (0xd2fc bytes) 2024-09-17T23:30:56.040 INFO:teuthology.orchestra.run.smithi139.stdout:5245 punch from 0x172ca00 to 0x1730e00, (0x4400 bytes) 2024-09-17T23:30:56.047 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:56.044+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:56.063 INFO:teuthology.orchestra.run.smithi037.stdout: marking collection for removal 2024-09-17T23:30:56.063 INFO:teuthology.orchestra.run.smithi037.stdout:setting '_remove' omap key 2024-09-17T23:30:56.064 INFO:teuthology.orchestra.run.smithi037.stdout:finish_remove_pgs 2.1_head removing 2.1 2024-09-17T23:30:56.065 INFO:teuthology.orchestra.run.smithi037.stdout:Remove successful 2024-09-17T23:30:56.089 INFO:teuthology.orchestra.run.smithi139.stdout:5246 read 0x59c6e00 thru 0x59cb49f (0x46a0 bytes) 2024-09-17T23:30:56.091 INFO:teuthology.orchestra.run.smithi139.stdout:5247 read 0x263e800 thru 0x264961f (0xae20 bytes) 2024-09-17T23:30:56.094 INFO:teuthology.orchestra.run.smithi139.stdout:5248 read 0x6d69a00 thru 0x6d7270a (0x8d0b bytes) 2024-09-17T23:30:56.098 INFO:teuthology.orchestra.run.smithi139.stdout:5249 trunc from 0xb5cda00 to 0x9029c00 2024-09-17T23:30:56.121 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:56.118+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:56.131 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:56.131 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:56.135 INFO:teuthology.orchestra.run.smithi139.stdout:5251 write 0x3b7a600 thru 0x3b867f8 (0xc1f9 bytes) 2024-09-17T23:30:56.137 INFO:teuthology.orchestra.run.smithi139.stdout:5252 read 0x655bc00 thru 0x656951e (0xd91f bytes) 2024-09-17T23:30:56.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:56.141 INFO:teuthology.orchestra.run.smithi139.stdout:5254 punch from 0x36d8200 to 0x36e2c00, (0xaa00 bytes) 2024-09-17T23:30:56.195 INFO:teuthology.orchestra.run.smithi139.stdout:5256 trunc from 0x9029c00 to 0x8a91200 2024-09-17T23:30:56.221 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:56.219+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:56.312 INFO:teuthology.orchestra.run.smithi139.stdout:5257 write 0x87a4200 thru 0x87a421e (0x1f bytes) 2024-09-17T23:30:56.314 INFO:teuthology.orchestra.run.smithi139.stdout:5258 read 0x3884800 thru 0x3884da2 (0x5a3 bytes) 2024-09-17T23:30:56.317 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:56.317 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:56.322 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:56.320+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:56.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:56.328 INFO:teuthology.orchestra.run.smithi139.stdout:5260 write 0x8dec200 thru 0x8df365c (0x745d bytes) 2024-09-17T23:30:56.328 INFO:teuthology.orchestra.run.smithi139.stdout:5263 write 0x1967200 thru 0x196be1f (0x4c20 bytes) 2024-09-17T23:30:56.330 INFO:teuthology.orchestra.run.smithi139.stdout:5264 write 0x76a00 thru 0x80298 (0x9899 bytes) 2024-09-17T23:30:56.330 INFO:teuthology.orchestra.run.smithi139.stdout:5266 read 0x11cf200 thru 0x11d0c75 (0x1a76 bytes) 2024-09-17T23:30:56.423 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:56.421+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:56.451 INFO:teuthology.orchestra.run.smithi139.stdout:5270 write 0x9b79600 thru 0x9b885fb (0xeffc bytes) 2024-09-17T23:30:56.457 INFO:teuthology.orchestra.run.smithi139.stdout:5271 read 0x6203400 thru 0x6204e98 (0x1a99 bytes) 2024-09-17T23:30:56.458 INFO:teuthology.orchestra.run.smithi139.stdout:5272 punch from 0x43be000 to 0x43c0800, (0x2800 bytes) 2024-09-17T23:30:56.463 INFO:teuthology.orchestra.run.smithi139.stdout:5275 read 0x8a5d800 thru 0x8a5df81 (0x782 bytes) 2024-09-17T23:30:56.468 INFO:teuthology.orchestra.run.smithi139.stdout:5277 write 0x3b71200 thru 0x3b77ac3 (0x68c4 bytes) 2024-09-17T23:30:56.480 INFO:teuthology.orchestra.run.smithi139.stdout:5281 trunc from 0x9b88600 to 0x1019800 2024-09-17T23:30:56.513 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:56.513 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:56.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:56.525 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:56.522+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:56.576 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.1.5 2024-09-17T23:30:56.607 INFO:teuthology.orchestra.run.smithi139.stdout:5282 punch from 0xee2600 to 0xef0400, (0xde00 bytes) 2024-09-17T23:30:56.609 INFO:teuthology.orchestra.run.smithi139.stdout:5283 read 0xa23000 thru 0xa2e11c (0xb11d bytes) 2024-09-17T23:30:56.625 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:56.623+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:56.638 INFO:teuthology.orchestra.run.smithi139.stdout:5289 write 0x7b2c000 thru 0x7b2fb4e (0x3b4f bytes) 2024-09-17T23:30:56.655 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:56.656 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:56.661 INFO:teuthology.orchestra.run.smithi139.stdout:5290 write 0xa77ea00 thru 0xa7843ff (0x5a00 bytes) 2024-09-17T23:30:56.661 INFO:teuthology.orchestra.run.smithi139.stdout:5291 write 0x9765600 thru 0x976e480 (0x8e81 bytes) 2024-09-17T23:30:56.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:56.701 INFO:teuthology.orchestra.run.smithi139.stdout:5292 write 0xd5ce000 thru 0xd5d8d5c (0xad5d bytes) 2024-09-17T23:30:56.702 INFO:teuthology.orchestra.run.smithi139.stdout:5293 write 0x216e400 thru 0x2170313 (0x1f14 bytes) 2024-09-17T23:30:56.702 INFO:teuthology.orchestra.run.smithi139.stdout:5296 read 0xd180a00 thru 0xd18dd44 (0xd345 bytes) 2024-09-17T23:30:56.702 INFO:teuthology.orchestra.run.smithi139.stdout:5297 write 0x853e00 thru 0x854575 (0x776 bytes) 2024-09-17T23:30:56.703 INFO:teuthology.orchestra.run.smithi139.stdout:5299 read 0x36aea00 thru 0x36b0f18 (0x2519 bytes) 2024-09-17T23:30:56.703 INFO:teuthology.orchestra.run.smithi139.stdout:5301 punch from 0x80ce400 to 0x80dda00, (0xf600 bytes) 2024-09-17T23:30:56.705 INFO:teuthology.orchestra.run.smithi139.stdout:5302 punch from 0xb377c00 to 0xb37ac00, (0x3000 bytes) 2024-09-17T23:30:56.705 INFO:teuthology.orchestra.run.smithi139.stdout:5303 read 0x9aeee00 thru 0x9af604c (0x724d bytes) 2024-09-17T23:30:56.705 INFO:teuthology.orchestra.run.smithi139.stdout:5304 write 0x3b8c200 thru 0x3b8dd66 (0x1b67 bytes) 2024-09-17T23:30:56.706 INFO:teuthology.orchestra.run.smithi139.stdout:5305 write 0x1b49200 thru 0x1b53f47 (0xad48 bytes) 2024-09-17T23:30:56.706 INFO:teuthology.orchestra.run.smithi139.stdout:5307 trunc from 0xd5d8e00 to 0xcb2e200 2024-09-17T23:30:56.726 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:56.724+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:56.778 INFO:teuthology.orchestra.run.smithi139.stdout:5309 write 0xb285000 thru 0xb293dd5 (0xedd6 bytes) 2024-09-17T23:30:56.779 INFO:teuthology.orchestra.run.smithi139.stdout:5310 read 0x25aee00 thru 0x25b5189 (0x638a bytes) 2024-09-17T23:30:56.779 INFO:teuthology.orchestra.run.smithi139.stdout:5311 read 0x9c40600 thru 0x9c4da1a (0xd41b bytes) 2024-09-17T23:30:56.780 INFO:teuthology.orchestra.run.smithi139.stdout:5312 read 0x9db2400 thru 0x9dbeda7 (0xc9a8 bytes) 2024-09-17T23:30:56.780 INFO:teuthology.orchestra.run.smithi139.stdout:5314 punch from 0xade200 to 0xae8600, (0xa400 bytes) 2024-09-17T23:30:56.800 INFO:teuthology.orchestra.run.smithi139.stdout:5317 read 0xbc07600 thru 0xbc11630 (0xa031 bytes) 2024-09-17T23:30:56.800 INFO:teuthology.orchestra.run.smithi139.stdout:5321 read 0xaa15600 thru 0xaa1a80f (0x5210 bytes) 2024-09-17T23:30:56.800 INFO:teuthology.orchestra.run.smithi139.stdout:5326 punch from 0x2600a00 to 0x260f000, (0xe600 bytes) 2024-09-17T23:30:56.801 INFO:teuthology.orchestra.run.smithi139.stdout:5328 write 0x553be00 thru 0x5540f59 (0x515a bytes) 2024-09-17T23:30:56.801 INFO:teuthology.orchestra.run.smithi139.stdout:5330 write 0x628c800 thru 0x6290a73 (0x4274 bytes) 2024-09-17T23:30:56.801 INFO:teuthology.orchestra.run.smithi139.stdout:5331 read 0x7590800 thru 0x759d856 (0xd057 bytes) 2024-09-17T23:30:56.808 INFO:teuthology.orchestra.run.smithi139.stdout:5332 trunc from 0xcb2e200 to 0x759e800 2024-09-17T23:30:56.827 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:56.824+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:56.849 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:56.849 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:56.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:56.909 INFO:teuthology.orchestra.run.smithi139.stdout:5333 read 0x94200 thru 0x94771 (0x572 bytes) 2024-09-17T23:30:56.910 INFO:teuthology.orchestra.run.smithi139.stdout:5334 read 0x59ed800 thru 0x59efcbb (0x24bc bytes) 2024-09-17T23:30:56.910 INFO:teuthology.orchestra.run.smithi139.stdout:5335 punch from 0x4be8600 to 0x4bf0600, (0x8000 bytes) 2024-09-17T23:30:56.910 INFO:teuthology.orchestra.run.smithi139.stdout:5336 read 0x333ec00 thru 0x33463d6 (0x77d7 bytes) 2024-09-17T23:30:56.910 INFO:teuthology.orchestra.run.smithi139.stdout:5338 read 0x1c9bc00 thru 0x1ca0470 (0x4871 bytes) 2024-09-17T23:30:56.911 INFO:teuthology.orchestra.run.smithi139.stdout:5339 read 0x148ac00 thru 0x1496065 (0xb466 bytes) 2024-09-17T23:30:56.911 INFO:teuthology.orchestra.run.smithi139.stdout:5341 write 0x26ce400 thru 0x26ce459 (0x5a bytes) 2024-09-17T23:30:56.944 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:56.942+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:56.946 INFO:teuthology.orchestra.run.smithi139.stdout:5342 write 0x887f200 thru 0x887f976 (0x777 bytes) 2024-09-17T23:30:56.946 INFO:teuthology.orchestra.run.smithi139.stdout:5343 write 0x75f4c00 thru 0x7603535 (0xe936 bytes) 2024-09-17T23:30:56.978 INFO:teuthology.orchestra.run.smithi139.stdout:5345 write 0x9615e00 thru 0x961ea0d (0x8c0e bytes) 2024-09-17T23:30:56.978 INFO:teuthology.orchestra.run.smithi139.stdout:5347 read 0x6f04e00 thru 0x6f0bffe (0x71ff bytes) 2024-09-17T23:30:56.979 INFO:teuthology.orchestra.run.smithi139.stdout:5348 read 0x253ee00 thru 0x25405c3 (0x17c4 bytes) 2024-09-17T23:30:56.989 INFO:teuthology.orchestra.run.smithi139.stdout:5349 trunc from 0x961ec00 to 0xfef200 2024-09-17T23:30:56.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:30:57.029 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:57.027+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:57.032 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.032 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:57.116 INFO:teuthology.orchestra.run.smithi139.stdout:5352 read 0xf7c000 thru 0xf82aec (0x6aed bytes) 2024-09-17T23:30:57.130 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:57.127+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:57.135 INFO:teuthology.orchestra.run.smithi139.stdout:5353 trunc from 0xfef200 to 0xdf0e000 2024-09-17T23:30:57.146 INFO:teuthology.orchestra.run.smithi139.stdout:5354 punch from 0xc46400 to 0xc53c00, (0xd800 bytes) 2024-09-17T23:30:57.164 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.164 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.167 INFO:teuthology.orchestra.run.smithi139.stdout:5356 read 0x9b42a00 thru 0x9b46c62 (0x4263 bytes) 2024-09-17T23:30:57.168 INFO:teuthology.orchestra.run.smithi139.stdout:5357 clone 36 order 23 su 4194304 sc 12 2024-09-17T23:30:57.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:57.233 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:57.230+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:57.296 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.296 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:57.348 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:57.346+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:57.411 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.411 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:57.435 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:57.433+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:57.535 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:57.533+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:57.570 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.570 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:30:57.636 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:57.634+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:57.689 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.689 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.697 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:30:57.741 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:57.739+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:57.804 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.805 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:30:57.857 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:57.854+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:57.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:30:57.924 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:30:57.924 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:30:57.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:30:57.938 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:57.937+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:57.976 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:30:58.027 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:30:58.027 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:30:58.039 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:58.036+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:58.044 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:30:58.044 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:30:58.054 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone35 from 0xdf0e000 (overlap 0xfef200) to 0x98f600 2024-09-17T23:30:58.064 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:30:58.064 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:30:58.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:30:58.092 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:30:58.108 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:58.109 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:58.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:30:58.110 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:30:58.110 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:30:58.114 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:30:58.114 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:30:58.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:30:58.160 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:58.157+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:58.241 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:58.238+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.425210970999998, 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:30:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:30:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:30:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.415638872000002, 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.412108365999998, 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.410613660000003, 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:30:58.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:30:58.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.410430740999999, 2024-09-17T23:30:58.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:30:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:30:58.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:30:58.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.410317239999998, 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:30:58.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.410293133000003, 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:30:58.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:30:58.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.410218802000003, 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:30:58.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:30:58.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.410195252999998, 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:30:58.309 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:30:58.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.407024051999997, 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:30:58.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.402615716, 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:30:58.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:30:58.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.375343608999998, 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:30:58.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:30:58.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:30:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.366161456999997, 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:58.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:58.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.366129964000002, 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:30:58.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:30:58.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.366104272000001, 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:30:58.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:30:58.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.365959719999999, 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:30:58.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.364391603000001, 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:30:58.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:30:58.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.364194255999998, 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:30:58.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:30:58.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.364167940999998, 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:30:58.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.364139598000001, 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:30:58.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.372 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:58.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:30:58.374 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:58.370+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:58.397 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T23:30:58.474 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:58.472+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:58.522 INFO:teuthology.orchestra.run.smithi037.stdout:get_pg_num_history pg_num_history pg_num_history(e93 pg_nums {1={12=1},2={15=8},3={19=32}} deleted_pools ) 2024-09-17T23:30:58.522 INFO:teuthology.orchestra.run.smithi037.stdout:Importing pgid 2.1 2024-09-17T23:30:58.522 INFO:teuthology.orchestra.run.smithi037.stdout:write_pg epoch 62 info 2.1( empty local-lis/les=15/16 n=0 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) 2024-09-17T23:30:58.527 INFO:teuthology.orchestra.run.smithi037.stdout:Import successful 2024-09-17T23:30:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:58.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:30:58.595 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:58.592+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:58.672 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:58.673 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:58.676 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:58.673+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:58.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:30:58.777 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:58.775+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:58.841 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.36718729899999, 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:30:58.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:30:58.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:58.851 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.509351486, 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:30:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.508659193, 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:30:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.508647612000004, 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:30:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.508120364999996, 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:30:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:30:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.508086382000002, 2024-09-17T23:30:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:30:58.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:30:58.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.507775789999997, 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.883 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:30:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.507749656000001, 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:30:58.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:30:58.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.507251749000005, 2024-09-17T23:30:58.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:30:58.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:30:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.506650659000002, 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:30:58.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.506102024, 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:30:58.904 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:58.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:30:58.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.505894425999998, 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:30:58.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.496278524999994, 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:30:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:30:58.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:30:58.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.496268291000007, 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:30:58.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.496258424000004, 2024-09-17T23:30:58.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:58.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:30:58.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.496248276000003, 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:30:58.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.495778408000007, 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:30:58.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:30:58.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.495756291999996, 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:30:58.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:30:58.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.495468596999999, 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.310082426999998, 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:30:58.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:58.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:30:58.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:30:58.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:58.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:30:58.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.959 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:58.959 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:58.959 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:58.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:30:58.961 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:30:58.958+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:30:58.972 INFO:teuthology.orchestra.run.smithi139.stdout:5358 read 0x32f7200 thru 0x330048e (0x928f bytes) 2024-09-17T23:30:58.973 INFO:teuthology.orchestra.run.smithi139.stdout:5360 read 0xadcd600 thru 0xadd06a7 (0x30a8 bytes) 2024-09-17T23:30:58.974 INFO:teuthology.orchestra.run.smithi139.stdout:5361 read 0x5868a00 thru 0x5878389 (0xf98a bytes) 2024-09-17T23:30:58.974 INFO:teuthology.orchestra.run.smithi139.stdout:5362 read 0x8d64000 thru 0x8d731bf (0xf1c0 bytes) 2024-09-17T23:30:58.975 INFO:teuthology.orchestra.run.smithi139.stdout:5364 read 0xddb1a00 thru 0xddb71ec (0x57ed bytes) 2024-09-17T23:30:58.975 INFO:teuthology.orchestra.run.smithi139.stdout:5367 write 0x483ca00 thru 0x48409fd (0x3ffe bytes) 2024-09-17T23:30:58.982 INFO:teuthology.orchestra.run.smithi139.stdout:5371 trunc from 0xdf0e000 to 0x8df9400 2024-09-17T23:30:59.023 INFO:teuthology.orchestra.run.smithi139.stdout:5373 trunc from 0x8df9400 to 0x8fba800 2024-09-17T23:30:59.038 INFO:teuthology.orchestra.run.smithi139.stdout:5375 read 0x28a9e00 thru 0x28b0e6a (0x706b bytes) 2024-09-17T23:30:59.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /var/log/ceph/exp.2.1.5 2024-09-17T23:30:59.049 INFO:teuthology.orchestra.run.smithi139.stdout:5378 trunc from 0x8fba800 to 0x10a800 2024-09-17T23:30:59.082 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:59.080+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:59.086 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:30:59.086 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:59.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:30:59.106 INFO:teuthology.orchestra.run.smithi139.stdout:5379 read 0x79c00 thru 0x7c9b4 (0x2db5 bytes) 2024-09-17T23:30:59.133 INFO:teuthology.orchestra.run.smithi139.stdout:5380 write 0xca3b400 thru 0xca3f09e (0x3c9f bytes) 2024-09-17T23:30:59.139 INFO:teuthology.orchestra.run.smithi139.stdout:5382 write 0xba98a00 thru 0xbaa002d (0x762e bytes) 2024-09-17T23:30:59.139 INFO:teuthology.orchestra.run.smithi139.stdout:5385 write 0x17c2c00 thru 0x17cbe26 (0x9227 bytes) 2024-09-17T23:30:59.139 INFO:teuthology.orchestra.run.smithi139.stdout:5386 read 0xa408400 thru 0xa4142b6 (0xbeb7 bytes) 2024-09-17T23:30:59.139 INFO:teuthology.orchestra.run.smithi139.stdout:5392 read 0x736fa00 thru 0x736fc47 (0x248 bytes) 2024-09-17T23:30:59.139 INFO:teuthology.orchestra.run.smithi139.stdout:5393 punch from 0x64f4200 to 0x64f4400, (0x200 bytes) 2024-09-17T23:30:59.154 INFO:teuthology.orchestra.run.smithi139.stdout:5394 write 0xebd2400 thru 0xebdf625 (0xd226 bytes) 2024-09-17T23:30:59.155 INFO:teuthology.orchestra.run.smithi139.stdout:5395 read 0x6cb000 thru 0x6cd83e (0x283f bytes) 2024-09-17T23:30:59.155 INFO:teuthology.orchestra.run.smithi139.stdout:5396 punch from 0x9ebfa00 to 0x9ecec00, (0xf200 bytes) 2024-09-17T23:30:59.156 INFO:teuthology.orchestra.run.smithi139.stdout:5397 punch from 0x7190e00 to 0x719bc00, (0xae00 bytes) 2024-09-17T23:30:59.162 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:59.159+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:59.172 INFO:teuthology.orchestra.run.smithi139.stdout:5398 trunc from 0xebdf800 to 0x7f2600 2024-09-17T23:30:59.200 INFO:teuthology.orchestra.run.smithi139.stdout:5401 trunc from 0x7f2600 to 0x8d53800 2024-09-17T23:30:59.207 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:30:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:30:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:30:59.208 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:59.208 INFO:teuthology.orchestra.run.smithi139.stdout:5405 punch from 0x15f9e00 to 0x15fe000, (0x4200 bytes) 2024-09-17T23:30:59.208 INFO:teuthology.orchestra.run.smithi139.stdout:5408 read 0x310c200 thru 0x3116ba6 (0xa9a7 bytes) 2024-09-17T23:30:59.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:30:59.223 INFO:teuthology.orchestra.run.smithi139.stdout:5409 write 0xe092400 thru 0xe093ef8 (0x1af9 bytes) 2024-09-17T23:30:59.224 INFO:teuthology.orchestra.run.smithi139.stdout:5410 read 0x8c15000 thru 0x8c190db (0x40dc bytes) 2024-09-17T23:30:59.225 INFO:teuthology.orchestra.run.smithi139.stdout:5412 read 0x493200 thru 0x4974b3 (0x42b4 bytes) 2024-09-17T23:30:59.225 INFO:teuthology.orchestra.run.smithi139.stdout:5415 read 0xc0d4a00 thru 0xc0d770c (0x2d0d bytes) 2024-09-17T23:30:59.225 INFO:teuthology.orchestra.run.smithi139.stdout:5419 punch from 0xbd08a00 to 0xbd14600, (0xbc00 bytes) 2024-09-17T23:30:59.225 INFO:teuthology.orchestra.run.smithi139.stdout:5420 read 0x820ca00 thru 0x820e3dc (0x19dd bytes) 2024-09-17T23:30:59.225 INFO:teuthology.orchestra.run.smithi139.stdout:5422 clone 37 order 22 su 2097152 sc 4 2024-09-17T23:30:59.302 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:59.299+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:59.364 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:59.362+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:59.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.327857334000001, 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:30:59.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:30:59.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.327684394000002, 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:30:59.372 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:30:59.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:30:59.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:59.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:59.378 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.327644523999993, 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:30:59.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:30:59.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.033140297000003, 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:30:59.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:30:59.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.785438748000004, 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:30:59.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:30:59.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:59.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.775506481999997, 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:30:59.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:30:59.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.762279433000003, 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:30:59.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:59.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221750276999998, 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:59.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221728886999998, 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:30:59.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:30:59.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221711757999998, 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:30:59.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:30:59.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221694476000003, 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:59.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:30:59.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221677446999998, 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:30:59.423 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:30:59.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:30:59.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.428 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221644572999999, 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:30:59.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.221056867000001, 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.434 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:30:59.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.199264726000003, 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:30:59.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.198727538, 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:30:59.443 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:30:59.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:30:59.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.198241422000002, 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:30:59.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.126098820000003, 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:30:59.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.126077008999999, 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:30:59.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:30:59.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.116634138999999, 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:30:59.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:30:59.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:59.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:30:59.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:30:59.468 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:30:59.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:30:59.470 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:59.467+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:59.571 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:30:59.569+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:30:59.578 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:59.578 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:59.578 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:30:59.578 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:59.587 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:30:59.672 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:30:59.670+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:30:59.694 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:59.695 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:30:59.695 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:30:59.695 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:30:59.695 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:59.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:30:59.773 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:30:59.771+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:30:59.852 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.974771465999993, 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:30:59.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.860 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.959146336000003, 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:30:59.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:30:59.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:30:59.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.95910868, 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.866 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:30:59.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:30:59.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:30:59.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.959091248999997, 2024-09-17T23:30:59.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:59.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:30:59.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.876 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.959056317999995, 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:30:59.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.959024897999996, 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:30:59.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:30:59.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:59.887 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.959008272000005, 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:59.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.892 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.958986577999994, 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:30:59.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:30:59.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 92.958974859999998, 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.899 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:30:59.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.999836340999998, 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:30:59.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:30:59.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:30:59.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995881177000001, 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.912 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:30:59.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:30:59.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.916 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995840518999998, 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:30:59.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.921 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995683784000001, 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:30:59.926 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995581094000002, 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.927 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:30:59.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995544871999996, 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.933 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:30:59.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.934 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:59.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:30:59.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:30:59.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:30:59.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:30:59.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995512577, 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.940 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:30:59.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:30:59.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 38.995357765999998, 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:30:59.945 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:30:59.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:30:59.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 33.995071445999997, 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.951 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:30:59.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:30:59.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:30:59.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:59.956 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 33.995002784999997, 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:30:59.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:30:59.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 33.994960810999999, 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:30:59.962 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:30:59.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:30:59.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.967 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:30:59.968 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:30:59.968 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:30:59.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:30:59.970 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:30:59.967+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:30:59.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:31:00.068 INFO:teuthology.orchestra.run.smithi139.stdout:flattening image image_client.0-clone36 2024-09-17T23:31:00.069 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:00.067+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:00.083 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:00.083 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:00.083 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:00.083 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:00.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:00.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:00.170 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:00.168+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:00.240 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:00.240 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:00.240 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:00.241 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:00.241 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:00.250 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:00.255 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:00.255 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:00.271 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:00.272 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:00.269+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:00.289 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:00.289 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:00.305 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:00.306 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:31:00.308 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:31:00.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:31:00.362 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:00.362 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:00.372 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:00.369+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:00.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:00.449 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T23:31:00.473 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:00.470+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:00.498 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:00.498 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:00.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:00.594 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:00.592+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:00.675 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:00.672+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:00.683 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:00.683 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:00.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:00.702 INFO:teuthology.orchestra.run.smithi139.stdout:5424 trunc from 0xe094000 to 0x5df7400 2024-09-17T23:31:00.732 INFO:teuthology.orchestra.run.smithi139.stdout:5425 trunc from 0x5df7400 to 0xe2e7e00 2024-09-17T23:31:00.742 INFO:teuthology.orchestra.run.smithi139.stdout:5428 read 0xaa30400 thru 0xaa338dd (0x34de bytes) 2024-09-17T23:31:00.743 INFO:teuthology.orchestra.run.smithi139.stdout:5429 read 0x2921400 thru 0x292251d (0x111e bytes) 2024-09-17T23:31:00.743 INFO:teuthology.orchestra.run.smithi139.stdout:5435 read 0x4264600 thru 0x426f73d (0xb13e bytes) 2024-09-17T23:31:00.743 INFO:teuthology.orchestra.run.smithi139.stdout:5438 punch from 0xe2c4a00 to 0xe2cf800, (0xae00 bytes) 2024-09-17T23:31:00.745 INFO:teuthology.orchestra.run.smithi139.stdout:5440 read 0xde7c200 thru 0xde8935a (0xd15b bytes) 2024-09-17T23:31:00.745 INFO:teuthology.orchestra.run.smithi139.stdout:5441 read 0x9176400 thru 0x9183bc9 (0xd7ca bytes) 2024-09-17T23:31:00.746 INFO:teuthology.orchestra.run.smithi139.stdout:5445 punch from 0x63b0e00 to 0x63b4600, (0x3800 bytes) 2024-09-17T23:31:00.746 INFO:teuthology.orchestra.run.smithi139.stdout:5447 read 0xac84000 thru 0xac9018d (0xc18e bytes) 2024-09-17T23:31:00.746 INFO:teuthology.orchestra.run.smithi139.stdout:5448 punch from 0x21d8600 to 0x21e2e00, (0xa800 bytes) 2024-09-17T23:31:00.746 INFO:teuthology.orchestra.run.smithi139.stdout:5450 punch from 0x6454400 to 0x645a800, (0x6400 bytes) 2024-09-17T23:31:00.746 INFO:teuthology.orchestra.run.smithi139.stdout:5451 write 0xad36800 thru 0xad3cdd2 (0x65d3 bytes) 2024-09-17T23:31:00.749 INFO:teuthology.orchestra.run.smithi139.stdout:5452 trunc from 0xe2e7e00 to 0xad72a00 2024-09-17T23:31:00.765 INFO:teuthology.orchestra.run.smithi139.stdout:5453 write 0xa5dd600 thru 0xa5e8e7e (0xb87f bytes) 2024-09-17T23:31:00.775 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:00.773+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:00.783 INFO:teuthology.orchestra.run.smithi139.stdout:5454 write 0xb8f8800 thru 0xb905c85 (0xd486 bytes) 2024-09-17T23:31:00.784 INFO:teuthology.orchestra.run.smithi139.stdout:5457 write 0x22a9800 thru 0x22ada99 (0x429a bytes) 2024-09-17T23:31:00.794 INFO:teuthology.orchestra.run.smithi139.stdout:5460 trunc from 0xb905e00 to 0x2a06c00 2024-09-17T23:31:00.827 INFO:teuthology.orchestra.run.smithi139.stdout:5461 trunc from 0x2a06c00 to 0x7340800 2024-09-17T23:31:00.842 INFO:teuthology.orchestra.run.smithi139.stdout:5462 read 0x68a8400 thru 0x68b3923 (0xb524 bytes) 2024-09-17T23:31:00.842 INFO:teuthology.orchestra.run.smithi139.stdout:5463 read 0x6c6c600 thru 0x6c716b5 (0x50b6 bytes) 2024-09-17T23:31:00.842 INFO:teuthology.orchestra.run.smithi139.stdout:5465 read 0x2fcc200 thru 0x2fd912f (0xcf30 bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5466 read 0x4d48000 thru 0x4d4c8fe (0x48ff bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5467 read 0xe3e200 thru 0xe42be2 (0x49e3 bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5469 punch from 0x41b7a00 to 0x41bb600, (0x3c00 bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5470 read 0x5734c00 thru 0x5736073 (0x1474 bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5471 read 0x137e800 thru 0x1381fc1 (0x37c2 bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5479 read 0x4704c00 thru 0x470aa2d (0x5e2e bytes) 2024-09-17T23:31:00.844 INFO:teuthology.orchestra.run.smithi139.stdout:5480 punch from 0x42c400 to 0x42ce00, (0xa00 bytes) 2024-09-17T23:31:00.846 INFO:teuthology.orchestra.run.smithi139.stdout:5481 trunc from 0x7340800 to 0xa2a6000 2024-09-17T23:31:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:00.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:00.864 INFO:teuthology.orchestra.run.smithi139.stdout:5483 read 0xcd2400 thru 0xcd5c5a (0x385b bytes) 2024-09-17T23:31:00.871 INFO:teuthology.orchestra.run.smithi139.stdout:5484 trunc from 0xa2a6000 to 0x4290c00 2024-09-17T23:31:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:5487 read 0x2637000 thru 0x2637374 (0x375 bytes) 2024-09-17T23:31:00.890 INFO:teuthology.orchestra.run.smithi139.stdout:5489 trunc from 0x4290c00 to 0x309a800 2024-09-17T23:31:00.905 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:00.902+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:00.940 INFO:teuthology.orchestra.run.smithi139.stdout:5491 write 0xc0c3e00 thru 0xc0cf60f (0xb810 bytes) 2024-09-17T23:31:00.940 INFO:teuthology.orchestra.run.smithi139.stdout:5493 read 0x95ec800 thru 0x95f8c0e (0xc40f bytes) 2024-09-17T23:31:00.940 INFO:teuthology.orchestra.run.smithi139.stdout:5494 read 0xb0a1800 thru 0xb0ab161 (0x9962 bytes) 2024-09-17T23:31:00.941 INFO:teuthology.orchestra.run.smithi139.stdout:5495 flatten 2024-09-17T23:31:00.963 INFO:teuthology.orchestra.run.smithi139.stdout:5496 punch from 0x3519c00 to 0x3524800, (0xac00 bytes) 2024-09-17T23:31:00.965 INFO:teuthology.orchestra.run.smithi139.stdout:5497 write 0xa772800 thru 0xa779b89 (0x738a bytes) 2024-09-17T23:31:00.965 INFO:teuthology.orchestra.run.smithi139.stdout:5498 read 0x4fe1200 thru 0x4feb916 (0xa717 bytes) 2024-09-17T23:31:00.965 INFO:teuthology.orchestra.run.smithi139.stdout:5501 write 0x3730200 thru 0x3736707 (0x6508 bytes) 2024-09-17T23:31:00.965 INFO:teuthology.orchestra.run.smithi139.stdout:5503 read 0xbe94e00 thru 0xbe9923b (0x443c bytes) 2024-09-17T23:31:00.965 INFO:teuthology.orchestra.run.smithi139.stdout:5504 read 0x69c1400 thru 0x69c9142 (0x7d43 bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5506 punch from 0x7dfe000 to 0x7e07400, (0x9400 bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5507 punch from 0x3a6be00 to 0x3a6ea00, (0x2c00 bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5508 read 0x7300400 thru 0x7300569 (0x16a bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5509 read 0x8af5000 thru 0x8b01705 (0xc706 bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5510 punch from 0xeee00 to 0xf9400, (0xa600 bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5512 write 0x1a3a800 thru 0x1a42c5b (0x845c bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5513 read 0xa11ae00 thru 0xa11e5c9 (0x37ca bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5515 write 0x4603e00 thru 0x460df8b (0xa18c bytes) 2024-09-17T23:31:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:5518 punch from 0xfa0800 to 0xfa3800, (0x3000 bytes) 2024-09-17T23:31:00.967 INFO:teuthology.orchestra.run.smithi139.stdout:5519 write 0x3765000 thru 0x376bdc3 (0x6dc4 bytes) 2024-09-17T23:31:00.967 INFO:teuthology.orchestra.run.smithi139.stdout:5520 read 0x57cba00 thru 0x57d6c72 (0xb273 bytes) 2024-09-17T23:31:00.967 INFO:teuthology.orchestra.run.smithi139.stdout:5521 read 0x4a0fa00 thru 0x4a15799 (0x5d9a bytes) 2024-09-17T23:31:00.977 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:00.975+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:00.981 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:00.981 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:00.991 INFO:teuthology.orchestra.run.smithi139.stdout:5522 write 0xe6a7800 thru 0xe6b7484 (0xfc85 bytes) 2024-09-17T23:31:00.991 INFO:teuthology.orchestra.run.smithi139.stdout:5523 write 0xa3e6a00 thru 0xa3e8874 (0x1e75 bytes) 2024-09-17T23:31:00.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:00.996 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:31:01.005 INFO:teuthology.orchestra.run.smithi139.stdout:5525 trunc from 0xe6b7600 to 0x2606800 2024-09-17T23:31:01.047 INFO:teuthology.orchestra.run.smithi139.stdout:5527 write 0x301b200 thru 0x302192f (0x6730 bytes) 2024-09-17T23:31:01.071 INFO:teuthology.orchestra.run.smithi139.stdout:5529 write 0xa76a400 thru 0xa7737f8 (0x93f9 bytes) 2024-09-17T23:31:01.078 INFO:teuthology.orchestra.run.smithi139.stdout:5530 write 0x846a200 thru 0x8475633 (0xb434 bytes) 2024-09-17T23:31:01.078 INFO:teuthology.orchestra.run.smithi139.stdout:5533 read 0x3eaae00 thru 0x3eb890d (0xdb0e bytes) 2024-09-17T23:31:01.078 INFO:teuthology.orchestra.run.smithi139.stdout:5534 clone 38 order 23 su 1048576 sc 13 2024-09-17T23:31:01.079 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:01.076+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:01.114 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.115 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:01.207 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:01.205+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:01.271 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.272 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:01.281 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:01.278+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:01.380 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:01.378+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:01.428 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.428 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:01.481 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:01.479+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:01.582 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:01.579+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:01.583 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.583 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:01.683 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:01.681+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.750 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:01.784 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:01.782+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:01.871 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.871 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.881 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:01.885 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:01.883+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:01.986 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:01.984+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:01.988 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:01.988 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:01.997 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:02.087 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:02.085+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:02.106 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:02.106 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:02.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:02.188 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:02.185+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:02.264 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:02.264 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:02.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:02.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:02.316 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:02.314+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:02.383 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:02.383 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:02.389 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:02.387+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:02.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:02.407 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T23:31:02.436 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:02.436 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:02.453 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:02.453 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:02.469 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:02.469 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:02.487 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:02.490 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:02.487+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:02.503 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:02.504 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:31:02.507 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:31:02.507 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:02.553 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.055999080999996, 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:02.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:02.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.055979201, 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.558 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:02.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.562 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.055957663000001, 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.563 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:02.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:02.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054075065000006, 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.567 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:02.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054064948999994, 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.572 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:02.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.576 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054052718999998, 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054042150000001, 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:02.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054032262000007, 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.586 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:02.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054021980000002, 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:02.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:02.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 85.054006092999998, 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.595 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:02.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:02.599 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 64.515080776000005, 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.600 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:02.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:02.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:02.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 64.513011390000003, 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:02.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 63.181144021999998, 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:02.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:02.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:02.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.696179344000001, 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.621 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.621 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.621 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:02.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:02.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:02.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.696146974000001, 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.629 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:02.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.695854597, 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:02.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:02.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.690030458999999, 2024-09-17T23:31:02.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:02.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.644 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.689988956000001, 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:02.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.647250689000003, 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:02.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:02.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.647148348000002, 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:02.661 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:02.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:02.663 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:02.660+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:02.763 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:02.761+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:02.771 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:02.772 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:02.772 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:02.772 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:02.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:02.864 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:02.862+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:02.928 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:02.928 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:02.928 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:02.929 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:02.929 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:02.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:02.966 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:02.964+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:02.971 INFO:teuthology.orchestra.run.smithi139.stdout:5536 trunc from 0xa773800 to 0x88bec00 2024-09-17T23:31:03.009 INFO:teuthology.orchestra.run.smithi139.stdout:5538 write 0x2183800 thru 0x2187a6d (0x426e bytes) 2024-09-17T23:31:03.010 INFO:teuthology.orchestra.run.smithi139.stdout:5539 write 0x2895c00 thru 0x289b66e (0x5a6f bytes) 2024-09-17T23:31:03.011 INFO:teuthology.orchestra.run.smithi139.stdout:5541 read 0x54bb200 thru 0x54bd4ad (0x22ae bytes) 2024-09-17T23:31:03.011 INFO:teuthology.orchestra.run.smithi139.stdout:5542 write 0x78c2200 thru 0x78d12a8 (0xf0a9 bytes) 2024-09-17T23:31:03.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:31:03.037 INFO:teuthology.orchestra.run.smithi139.stdout:5545 write 0xe005200 thru 0xe00d78a (0x858b bytes) 2024-09-17T23:31:03.049 INFO:teuthology.orchestra.run.smithi139.stdout:5546 write 0xe48e400 thru 0xe4937dc (0x53dd bytes) 2024-09-17T23:31:03.055 INFO:teuthology.orchestra.run.smithi139.stdout:5549 write 0xd0ff600 thru 0xd10eeb8 (0xf8b9 bytes) 2024-09-17T23:31:03.063 INFO:teuthology.orchestra.run.smithi139.stdout:5552 trunc from 0xe493800 to 0x7d3a000 2024-09-17T23:31:03.066 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:03.063+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:03.089 INFO:teuthology.orchestra.run.smithi139.stdout:5555 write 0x76ee400 thru 0x76f79e0 (0x95e1 bytes) 2024-09-17T23:31:03.103 INFO:teuthology.orchestra.run.smithi139.stdout:5556 write 0x9e93200 thru 0x9e96e8b (0x3c8c bytes) 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.262690036999999, 2024-09-17T23:31:03.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:03.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:03.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:03.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.253117938000003, 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:03.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.249587431999998, 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.248092726000003, 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:03.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:03.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.247909806999999, 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:03.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.247796305999998, 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:03.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:03.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.247772199000003, 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:03.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.150 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.247697868000003, 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:03.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:03.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.247674318999998, 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:03.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.244503117999997, 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:03.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.240094782, 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:03.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:03.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:03.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.212822674999998, 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:03.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:03.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:03.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:03.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:03.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.203640522999997, 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:03.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:03.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:03.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.203609030000003, 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:03.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:03.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.203583338000001, 2024-09-17T23:31:03.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:03.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.203438786, 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:03.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:03.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.201870669000002, 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:03.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.201673321999998, 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:03.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:03.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.201647006999998, 2024-09-17T23:31:03.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:03.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:03.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.201618664000002, 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:03.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.221 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:03.222 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:03.220+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:03.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5558 trunc from 0x9e97000 to 0x4959400 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5559 write 0x6818400 thru 0x681861e (0x21f bytes) 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5560 write 0x7eaa400 thru 0x7eafe49 (0x5a4a bytes) 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5562 write 0xcad2c00 thru 0xcade25e (0xb65f bytes) 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5563 trunc from 0xcade400 to 0xc37be00 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5565 punch from 0x8a8a600 to 0x8a95e00, (0xb800 bytes) 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5568 punch from 0xbc3e800 to 0xbc41200, (0x2a00 bytes) 2024-09-17T23:31:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:5569 write 0xa7ffe00 thru 0xa802c9c (0x2e9d bytes) 2024-09-17T23:31:03.225 INFO:teuthology.orchestra.run.smithi139.stdout:5573 punch from 0x7d10a00 to 0x7d12200, (0x1800 bytes) 2024-09-17T23:31:03.225 INFO:teuthology.orchestra.run.smithi139.stdout:5575 write 0x2643a00 thru 0x2648ed9 (0x54da bytes) 2024-09-17T23:31:03.225 INFO:teuthology.orchestra.run.smithi139.stdout:5576 read 0x62f6c00 thru 0x62feec5 (0x82c6 bytes) 2024-09-17T23:31:03.225 INFO:teuthology.orchestra.run.smithi139.stdout:5577 trunc from 0xc37be00 to 0xa652a00 2024-09-17T23:31:03.234 INFO:teuthology.orchestra.run.smithi139.stdout:5580 read 0xf83e00 thru 0xf87e3a (0x403b bytes) 2024-09-17T23:31:03.235 INFO:teuthology.orchestra.run.smithi139.stdout:5581 read 0x506a400 thru 0x50788d7 (0xe4d8 bytes) 2024-09-17T23:31:03.236 INFO:teuthology.orchestra.run.smithi139.stdout:5582 write 0x32b5200 thru 0x32bbdb2 (0x6bb3 bytes) 2024-09-17T23:31:03.237 INFO:teuthology.orchestra.run.smithi139.stdout:5583 trunc from 0xa652a00 to 0xb59b600 2024-09-17T23:31:03.254 INFO:teuthology.orchestra.run.smithi139.stdout:5585 punch from 0x2d65800 to 0x2d6ce00, (0x7600 bytes) 2024-09-17T23:31:03.254 INFO:teuthology.orchestra.run.smithi139.stdout:5587 read 0x8cfa00 thru 0x8db793 (0xbd94 bytes) 2024-09-17T23:31:03.272 INFO:teuthology.orchestra.run.smithi139.stdout:5588 write 0xd197400 thru 0xd19befa (0x4afb bytes) 2024-09-17T23:31:03.272 INFO:teuthology.orchestra.run.smithi139.stdout:5592 write 0xe17600 thru 0xe17e2f (0x830 bytes) 2024-09-17T23:31:03.273 INFO:teuthology.orchestra.run.smithi139.stdout:5593 read 0xc7a1400 thru 0xc7a2652 (0x1253 bytes) 2024-09-17T23:31:03.273 INFO:teuthology.orchestra.run.smithi139.stdout:5594 read 0xc90f800 thru 0xc91e1c5 (0xe9c6 bytes) 2024-09-17T23:31:03.273 INFO:teuthology.orchestra.run.smithi139.stdout:5595 flatten 2024-09-17T23:31:03.296 INFO:teuthology.orchestra.run.smithi139.stdout:5596 read 0xd059e00 thru 0xd063046 (0x9247 bytes) 2024-09-17T23:31:03.297 INFO:teuthology.orchestra.run.smithi139.stdout:5597 read 0xb5b1600 thru 0xb5b68da (0x52db bytes) 2024-09-17T23:31:03.302 INFO:teuthology.orchestra.run.smithi139.stdout:5600 trunc from 0xd19c000 to 0x8cd1200 2024-09-17T23:31:03.322 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:03.320+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:03.329 INFO:teuthology.orchestra.run.smithi139.stdout:5601 punch from 0x4aa1200 to 0x4aade00, (0xcc00 bytes) 2024-09-17T23:31:03.329 INFO:teuthology.orchestra.run.smithi139.stdout:5603 read 0x3e94c00 thru 0x3ea4af2 (0xfef3 bytes) 2024-09-17T23:31:03.348 INFO:teuthology.orchestra.run.smithi139.stdout:5604 write 0xb7db800 thru 0xb7e2180 (0x6981 bytes) 2024-09-17T23:31:03.349 INFO:teuthology.orchestra.run.smithi139.stdout:5606 punch from 0x1eb9c00 to 0x1ebc800, (0x2c00 bytes) 2024-09-17T23:31:03.350 INFO:teuthology.orchestra.run.smithi139.stdout:5607 punch from 0x2e6b200 to 0x2e78a00, (0xd800 bytes) 2024-09-17T23:31:03.350 INFO:teuthology.orchestra.run.smithi139.stdout:5608 read 0x6161800 thru 0x617078f (0xef90 bytes) 2024-09-17T23:31:03.350 INFO:teuthology.orchestra.run.smithi139.stdout:5612 read 0xab41000 thru 0xab4ee76 (0xde77 bytes) 2024-09-17T23:31:03.351 INFO:teuthology.orchestra.run.smithi139.stdout:5613 punch from 0x30bae00 to 0x30c1e00, (0x7000 bytes) 2024-09-17T23:31:03.357 INFO:teuthology.orchestra.run.smithi139.stdout:5615 trunc from 0xb7e2200 to 0x6142e00 2024-09-17T23:31:03.379 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:03.379 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:03.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:03.396 INFO:teuthology.orchestra.run.smithi139.stdout:5617 write 0xda36200 thru 0xda40eb7 (0xacb8 bytes) 2024-09-17T23:31:03.402 INFO:teuthology.orchestra.run.smithi139.stdout:5618 read 0xc1d2200 thru 0xc1e0f43 (0xed44 bytes) 2024-09-17T23:31:03.411 INFO:teuthology.orchestra.run.smithi139.stdout:5620 trunc from 0xda41000 to 0x6d80000 2024-09-17T23:31:03.427 INFO:teuthology.orchestra.run.smithi139.stdout:5621 punch from 0x23fe200 to 0x23ff400, (0x1200 bytes) 2024-09-17T23:31:03.433 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:03.430+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:03.439 INFO:teuthology.orchestra.run.smithi139.stdout:5623 trunc from 0x6d80000 to 0xaac0200 2024-09-17T23:31:03.467 INFO:teuthology.orchestra.run.smithi139.stdout:5625 write 0xce4a200 thru 0xce519ad (0x77ae bytes) 2024-09-17T23:31:03.474 INFO:teuthology.orchestra.run.smithi139.stdout:5626 write 0x393c600 thru 0x393ea1d (0x241e bytes) 2024-09-17T23:31:03.476 INFO:teuthology.orchestra.run.smithi139.stdout:5628 read 0xabd4200 thru 0xabdfe04 (0xbc05 bytes) 2024-09-17T23:31:03.476 INFO:teuthology.orchestra.run.smithi139.stdout:5632 write 0x8433a00 thru 0x843cf13 (0x9514 bytes) 2024-09-17T23:31:03.476 INFO:teuthology.orchestra.run.smithi139.stdout:5633 read 0x8af1200 thru 0x8af2a18 (0x1819 bytes) 2024-09-17T23:31:03.476 INFO:teuthology.orchestra.run.smithi139.stdout:5634 write 0xc263200 thru 0xc26b80a (0x860b bytes) 2024-09-17T23:31:03.476 INFO:teuthology.orchestra.run.smithi139.stdout:5636 write 0x1173e00 thru 0x1177a6c (0x3c6d bytes) 2024-09-17T23:31:03.477 INFO:teuthology.orchestra.run.smithi139.stdout:5637 clone 39 order 20 su 16384 sc 15 2024-09-17T23:31:03.503 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:03.503 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:03.503 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:03.503 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:03.503 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:03.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:03.553 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:03.550+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:03.625 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:03.623+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:03.662 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:03.662 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:03.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:03.662 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 124.187571417, 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:03.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:03.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:03.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:03.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.329735604000007, 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.329043311000007, 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:03.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.329031729999997, 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.328504483000003, 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:03.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:03.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.328470499999995, 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:03.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:03.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:03.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:03.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.328159908000003, 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.328133773999994, 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:03.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:03.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.327635866999998, 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:03.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.327034776999994, 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:31:03.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:03.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.326486141999993, 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:03.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:03.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.326278544000004, 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:03.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:03.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.316662643000001, 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:03.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.316652409, 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:03.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.316642541999997, 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:03.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.316632393999996, 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:03.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:03.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:03.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:03.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.316162525999999, 2024-09-17T23:31:03.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:03.747 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:03.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:03.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:03.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:03.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:03.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:03.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.316140410000003, 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:03.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.315852715000005, 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:31:03.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.130466545000004, 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:03.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:03.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:03.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:03.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:03.769 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:03.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:03.771 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:03.768+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:03.872 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:03.870+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:03.919 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:03.919 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:03.919 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:03.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:03.973 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:03.970+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:04.011 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:31:04.038 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:04.038 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:04.038 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:04.038 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:04.038 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:04.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:04.089 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:04.086+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:04.108 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [5] live_osds: [0, 1, 2, 3, 4] 2024-09-17T23:31:04.108 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:04.108 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-17T23:31:04.109 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T23:31:04.109 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T23:31:04.111 INFO:tasks.ceph.osd.5:Started 2024-09-17T23:31:04.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:04.174 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:04.171+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:04.196 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:04.196 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.157867371999998, 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:04.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:04.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:04.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.157694432, 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:04.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:04.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.208 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.157654562000005, 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:04.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:04.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.863150335, 2024-09-17T23:31:04.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:04.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:04.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.615448786000002, 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:04.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.605516519999995, 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:04.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.592289471000001, 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051760314999996, 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051738924999995, 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:04.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:04.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051721795999995, 2024-09-17T23:31:04.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:04.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:04.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051704513999994, 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:04.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:04.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051687485000002, 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:04.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:04.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051654611000004, 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:04.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.051066904999999, 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:04.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.029274764, 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:04.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:04.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.028737575999997, 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:04.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:04.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.02825146, 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.956108858, 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:04.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:04.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:04.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:04.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.287 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.956087046999997, 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:04.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:04.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.946644177000003, 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:04.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:04.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:04.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:04.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:04.298 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:04.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T23:31:04.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:no valid command found; 10 closest matches: 2024-09-17T23:31:04.300 INFO:teuthology.orchestra.run.smithi037.stderr:0 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:1 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:2 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:abort 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:assert 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:config diff 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:config diff get 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:config get 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:config help [] 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:config set ... 2024-09-17T23:31:04.301 INFO:teuthology.orchestra.run.smithi037.stderr:admin_socket: invalid command 2024-09-17T23:31:04.301 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-17T23:31:04.301 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:04.297+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:04.400 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:04.398+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:04.423 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T23:31:04.449 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:04.450 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:04.450 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:04.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:04.501 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:04.498+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:04.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:04.602 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:04.600+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:04.613 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:04.613 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:04.613 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:04.613 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:04.613 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:04.623 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:04.644 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:04.666 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:04.666 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:04.682 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:04.683 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:04.701 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:04.701 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:04.707 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:04.705+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:04.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:04.718 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:04.719 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:31:04.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:31:04.722 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:31:04.773 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:04.774 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:04.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:04.804 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:04.802+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:04.884 INFO:teuthology.orchestra.run.smithi139.stdout:5640 write 0x3948200 thru 0x394b6ce (0x34cf bytes) 2024-09-17T23:31:04.885 INFO:teuthology.orchestra.run.smithi139.stdout:5641 read 0xc83200 thru 0xc9060d (0xd40e bytes) 2024-09-17T23:31:04.887 INFO:teuthology.orchestra.run.smithi139.stdout:5642 punch from 0xb32f800 to 0xb33ba00, (0xc200 bytes) 2024-09-17T23:31:04.904 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:04.902+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:04.918 INFO:teuthology.orchestra.run.smithi139.stdout:5644 trunc from 0xce51a00 to 0x3be7800 2024-09-17T23:31:04.946 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:04.946 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:04.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:04.960 INFO:teuthology.orchestra.run.smithi139.stdout:5647 read 0x8e4a00 thru 0x8ed940 (0x8f41 bytes) 2024-09-17T23:31:04.962 INFO:teuthology.orchestra.run.smithi139.stdout:5649 punch from 0x152da00 to 0x1536e00, (0x9400 bytes) 2024-09-17T23:31:04.989 INFO:teuthology.orchestra.run.smithi139.stdout:5650 write 0x8aa4800 thru 0x8aaa453 (0x5c54 bytes) 2024-09-17T23:31:04.989 INFO:teuthology.orchestra.run.smithi139.stdout:5652 read 0x8972e00 thru 0x897b393 (0x8594 bytes) 2024-09-17T23:31:04.989 INFO:teuthology.orchestra.run.smithi139.stdout:5654 punch from 0x5348800 to 0x534ee00, (0x6600 bytes) 2024-09-17T23:31:04.991 INFO:teuthology.orchestra.run.smithi139.stdout:5655 clone 40 order 19 su 65536 sc 10 2024-09-17T23:31:05.042 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:05.040+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:05.093 INFO:teuthology.orchestra.run.smithi139.stdout:flattening image image_client.0-clone39 2024-09-17T23:31:05.107 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:05.105+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:05.124 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:05.125 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:05.140 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:05.226 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:05.224+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:05.309 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:05.307+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:05.334 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:05.334 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:05.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:05.409 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:05.407+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:05.452 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:05.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:05.545 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:05.542+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:05.611 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:05.609+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:05.612 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:05.612 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:05.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:05.656 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-17T23:31:05.712 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:05.709+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:05.769 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:05.769 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:05.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:05.812 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:05.810+0000 7f6a30a2f640 -1 Falling back to public interface 2024-09-17T23:31:05.820 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:05.818+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:05.913 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:05.911+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:05.931 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:05.932 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:05.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:06.025 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:06.022+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:06.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:31:06.092 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:06.092 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:06.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:06.144 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:06.142+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:06.210 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:06.210 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:06.216 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:06.215+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:06.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:06.229 INFO:teuthology.orchestra.run.smithi139.stdout:5656 clone 41 order 18 su 32768 sc 4 2024-09-17T23:31:06.317 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:06.314+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:06.365 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:06.366 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:06.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:06.418 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:06.415+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:06.518 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:06.515+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:06.521 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:06.521 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:06.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:06.619 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:06.617+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:06.678 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:06.678 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:06.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:06.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:06.731 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:06.729+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:06.807 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:06.807 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:06.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:06.821 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:06.818+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:06.849 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:06.849 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:06.883 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:06.883 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:06.900 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:06.900 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:06.917 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:06.917 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:06.917 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:31:06.920 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:31:06.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:31:06.922 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:06.920+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:06.966 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:07.014 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:07.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:07.022 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:07.019+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:07.028 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:31:07.099 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone40 from 0x8aaa600 (overlap 0x8aaa600) to 0x4555200 2024-09-17T23:31:07.126 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:07.123+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:07.199 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.321392113, 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.200 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:07.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:07.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.205 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.305766983, 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:07.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.30572932699999, 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:07.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:07.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:07.216 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.30571189600001, 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:07.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:07.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.221 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.305676965, 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:07.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.226 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.305645545, 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:07.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.231 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.305628919, 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.236 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.305607225, 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:07.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.30559550700001, 2024-09-17T23:31:07.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:07.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:07.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.346456988, 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:07.247 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:07.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:07.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.342501824000003, 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:07.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:07.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.342461166, 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:07.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.342304431000002, 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:31:07.269 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:07.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:31:07.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.274 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.342201740999997, 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.275 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:07.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:07.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:07.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.279 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.342165518999998, 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.284 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.342133224000001, 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:07.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:07.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:07.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.289 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.341978413, 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:07.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:07.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.341692092999999, 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.341623431999999, 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.300 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:07.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:07.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.341581458, 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:07.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.310 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:07.312 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:07.312 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:07.309+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:07.412 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:07.410+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:07.461 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:07.462 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:07.462 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:07.462 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:07.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:07.496 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-17T23:31:07.512 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:07.510+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:07.613 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:07.611+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:07.664 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:07.664 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:07.664 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:07.665 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:07.665 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:07.676 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:07.713 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:07.711+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:07.815 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:07.812+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:07.868 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:07.868 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:07.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:07.868 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.371065724000005, 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.869 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:07.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.371045843999994, 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:07.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.371024305999995, 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.878 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.879 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:07.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369141708000001, 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.883 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.885 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:07.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369131592000002, 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:07.887 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:07.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:07.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369119362000006, 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.892 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:07.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:07.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:07.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369108792999995, 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:07.896 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:07.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:07.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369098905000001, 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.901 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:07.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:07.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369088622999996, 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.905 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:07.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.909 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 90.369072736000007, 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:07.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 69.830147418999999, 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:07.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:07.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 69.828078032999997, 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:07.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.496210665000007, 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:07.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 47.011245987000002, 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:07.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:07.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 47.011213617000003, 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.935 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:07.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:07.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:07.940 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 47.010921240000002, 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:07.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.945 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 47.005097102000001, 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:07.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 47.005055599000002, 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.951 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:07.955 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.962317331999998, 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:07.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:07.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:07.960 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.962214991000003, 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:07.961 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:07.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:07.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:07.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:07.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:07.966 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:07.967 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:07.965+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:07.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:08.068 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:08.065+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:08.077 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:08.078 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:08.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:08.137 INFO:teuthology.orchestra.run.smithi139.stdout:5657 trunc from 0x8aaa600 to 0x190ac00 2024-09-17T23:31:08.158 INFO:teuthology.orchestra.run.smithi139.stdout:5660 punch from 0x893400 to 0x895e00, (0x2a00 bytes) 2024-09-17T23:31:08.170 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:08.167+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:08.179 INFO:teuthology.orchestra.run.smithi139.stdout:5661 write 0x5b2be00 thru 0x5b3652c (0xa72d bytes) 2024-09-17T23:31:08.180 INFO:teuthology.orchestra.run.smithi139.stdout:5662 punch from 0x4354000 to 0x4354600, (0x600 bytes) 2024-09-17T23:31:08.182 INFO:teuthology.orchestra.run.smithi139.stdout:5663 punch from 0x3cb3000 to 0x3cb6600, (0x3600 bytes) 2024-09-17T23:31:08.182 INFO:teuthology.orchestra.run.smithi139.stdout:5665 write 0x4eed400 thru 0x4ef3c31 (0x6832 bytes) 2024-09-17T23:31:08.183 INFO:teuthology.orchestra.run.smithi139.stdout:5666 read 0x42cfa00 thru 0x42d7d96 (0x8397 bytes) 2024-09-17T23:31:08.193 INFO:teuthology.orchestra.run.smithi139.stdout:5667 trunc from 0x5b36600 to 0xdfc3800 2024-09-17T23:31:08.205 INFO:teuthology.orchestra.run.smithi139.stdout:5669 write 0x7532c00 thru 0x753bd81 (0x9182 bytes) 2024-09-17T23:31:08.211 INFO:teuthology.orchestra.run.smithi139.stdout:5672 read 0x1aba000 thru 0x1abc20b (0x220c bytes) 2024-09-17T23:31:08.231 INFO:teuthology.orchestra.run.smithi139.stdout:5679 write 0xe118400 thru 0xe123f25 (0xbb26 bytes) 2024-09-17T23:31:08.231 INFO:teuthology.orchestra.run.smithi139.stdout:5680 read 0x6c71c00 thru 0x6c74a1b (0x2e1c bytes) 2024-09-17T23:31:08.231 INFO:teuthology.orchestra.run.smithi139.stdout:5685 write 0x761ae00 thru 0x761c9a1 (0x1ba2 bytes) 2024-09-17T23:31:08.237 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:08.237 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:08.237 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:08.237 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:08.237 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:08.244 INFO:teuthology.orchestra.run.smithi139.stdout:5686 trunc from 0xe124000 to 0x30bae00 2024-09-17T23:31:08.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:08.275 INFO:teuthology.orchestra.run.smithi139.stdout:5690 write 0x8d600 thru 0x9470c (0x710d bytes) 2024-09-17T23:31:08.275 INFO:teuthology.orchestra.run.smithi139.stdout:5691 read 0x1b84a00 thru 0x1b86933 (0x1f34 bytes) 2024-09-17T23:31:08.289 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:08.287+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:08.292 INFO:teuthology.orchestra.run.smithi139.stdout:5693 write 0x50c5200 thru 0x50d3253 (0xe054 bytes) 2024-09-17T23:31:08.299 INFO:teuthology.orchestra.run.smithi139.stdout:5694 punch from 0x2972800 to 0x2976e00, (0x4600 bytes) 2024-09-17T23:31:08.302 INFO:teuthology.orchestra.run.smithi139.stdout:5695 read 0x25d0600 thru 0x25d9ea7 (0x98a8 bytes) 2024-09-17T23:31:08.302 INFO:teuthology.orchestra.run.smithi139.stdout:5698 write 0x536600 thru 0x536c59 (0x65a bytes) 2024-09-17T23:31:08.302 INFO:teuthology.orchestra.run.smithi139.stdout:5702 punch from 0x19bde00 to 0x19ca400, (0xc600 bytes) 2024-09-17T23:31:08.302 INFO:teuthology.orchestra.run.smithi139.stdout:5703 read 0x2f8fc00 thru 0x2f91b68 (0x1f69 bytes) 2024-09-17T23:31:08.321 INFO:teuthology.orchestra.run.smithi139.stdout:5706 write 0x8686a00 thru 0x8695940 (0xef41 bytes) 2024-09-17T23:31:08.336 INFO:teuthology.orchestra.run.smithi139.stdout:5708 trunc from 0x8695a00 to 0x1a2b200 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.513489387, 2024-09-17T23:31:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.503917287999997, 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.500386782, 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.498892075999997, 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.498709157, 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.498595655999999, 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.498571548999998, 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.498497217999997, 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.498473668999999, 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.495302467999998, 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.490894132000001, 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.463622024999999, 2024-09-17T23:31:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.454439872999998, 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.454408379999997, 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.454382688000003, 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.454238136000001, 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.452670019000003, 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.452472671999999, 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.452446356999999, 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.452418014000003, 2024-09-17T23:31:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.463 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:08.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:08.466 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:08.462+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:08.467 INFO:teuthology.orchestra.run.smithi139.stdout:5710 write 0xd7a6a00 thru 0xd7b2df1 (0xc3f2 bytes) 2024-09-17T23:31:08.467 INFO:teuthology.orchestra.run.smithi139.stdout:5712 read 0xba26400 thru 0xba2888b (0x248c bytes) 2024-09-17T23:31:08.467 INFO:teuthology.orchestra.run.smithi139.stdout:5713 write 0xca07000 thru 0xca11ecc (0xaecd bytes) 2024-09-17T23:31:08.467 INFO:teuthology.orchestra.run.smithi139.stdout:5714 read 0x3c0e000 thru 0x3c12523 (0x4524 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5715 read 0x5376a00 thru 0x53852b6 (0xe8b7 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5716 punch from 0x1f67600 to 0x1f73400, (0xbe00 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5719 read 0x6a70400 thru 0x6a71604 (0x1205 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5721 punch from 0x5cd5600 to 0x5ce3800, (0xe200 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5723 write 0xe378e00 thru 0xe387547 (0xe748 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5727 write 0xa361e00 thru 0xa3669bb (0x4bbc bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5728 write 0x2aea600 thru 0x2aebffb (0x19fc bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5729 read 0x28e7200 thru 0x28e7464 (0x265 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5731 punch from 0xadc5000 to 0xadcb600, (0x6600 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5732 trunc from 0xe387600 to 0x4f31c00 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5733 write 0x9cbd400 thru 0x9cc94f2 (0xc0f3 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5734 write 0x5309e00 thru 0x530ee68 (0x5069 bytes) 2024-09-17T23:31:08.468 INFO:teuthology.orchestra.run.smithi139.stdout:5735 punch from 0x8141e00 to 0x814fc00, (0xde00 bytes) 2024-09-17T23:31:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:5740 write 0x4530000 thru 0x453d88f (0xd890 bytes) 2024-09-17T23:31:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:5742 write 0x402e800 thru 0x4034870 (0x6071 bytes) 2024-09-17T23:31:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:5743 write 0x9b2c00 thru 0x9bf5fb (0xc9fc bytes) 2024-09-17T23:31:08.469 INFO:teuthology.orchestra.run.smithi139.stdout:5745 trunc from 0x9cc9600 to 0xa7a0800 2024-09-17T23:31:08.493 INFO:teuthology.orchestra.run.smithi139.stdout:5746 trunc from 0xa7a0800 to 0x9e23800 2024-09-17T23:31:08.506 INFO:teuthology.orchestra.run.smithi139.stdout:5747 read 0xb8e800 thru 0xb8eb2c (0x32d bytes) 2024-09-17T23:31:08.507 INFO:teuthology.orchestra.run.smithi139.stdout:5751 read 0x1bad800 thru 0x1bbb7ec (0xdfed bytes) 2024-09-17T23:31:08.507 INFO:teuthology.orchestra.run.smithi139.stdout:5753 read 0x2fd8200 thru 0x2fe6fc0 (0xedc1 bytes) 2024-09-17T23:31:08.514 INFO:teuthology.orchestra.run.smithi139.stdout:5757 trunc from 0x9e23800 to 0x42c8200 2024-09-17T23:31:08.532 INFO:teuthology.orchestra.run.smithi139.stdout:5758 punch from 0x2be6200 to 0x2bf0000, (0x9e00 bytes) 2024-09-17T23:31:08.549 INFO:teuthology.orchestra.run.smithi139.stdout:5761 write 0x80aa200 thru 0x80b7ec5 (0xdcc6 bytes) 2024-09-17T23:31:08.555 INFO:teuthology.orchestra.run.smithi139.stdout:5762 punch from 0x1b8d200 to 0x1b8d400, (0x200 bytes) 2024-09-17T23:31:08.557 INFO:teuthology.orchestra.run.smithi139.stdout:5763 read 0x6381200 thru 0x6386a46 (0x5847 bytes) 2024-09-17T23:31:08.557 INFO:teuthology.orchestra.run.smithi139.stdout:5764 punch from 0x1a48200 to 0x1a52c00, (0xaa00 bytes) 2024-09-17T23:31:08.558 INFO:teuthology.orchestra.run.smithi139.stdout:5766 punch from 0x145e00 to 0x14ce00, (0x7000 bytes) 2024-09-17T23:31:08.566 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:08.564+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:08.573 INFO:teuthology.orchestra.run.smithi139.stdout:5767 write 0xba21800 thru 0xba26524 (0x4d25 bytes) 2024-09-17T23:31:08.574 INFO:teuthology.orchestra.run.smithi139.stdout:5770 read 0x8581400 thru 0x8590dd2 (0xf9d3 bytes) 2024-09-17T23:31:08.574 INFO:teuthology.orchestra.run.smithi139.stdout:5777 punch from 0x811f000 to 0x8126a00, (0x7a00 bytes) 2024-09-17T23:31:08.576 INFO:teuthology.orchestra.run.smithi139.stdout:5781 read 0x134f800 thru 0x135d430 (0xdc31 bytes) 2024-09-17T23:31:08.579 INFO:teuthology.orchestra.run.smithi139.stdout:5786 punch from 0x5667400 to 0x566aa00, (0x3600 bytes) 2024-09-17T23:31:08.579 INFO:teuthology.orchestra.run.smithi139.stdout:5788 read 0x268c800 thru 0x2692ce4 (0x64e5 bytes) 2024-09-17T23:31:08.579 INFO:teuthology.orchestra.run.smithi139.stdout:5790 clone 42 order 21 su 32768 sc 15 2024-09-17T23:31:08.617 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:08.617 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:08.617 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:08.618 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:08.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:08.667 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:08.665+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:08.734 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:08.734 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:08.734 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:08.734 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:08.734 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:08.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:08.786 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:08.785+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:08.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 129.37746681199999, 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:08.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:08.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.519630999, 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:08.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:08.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.518938706, 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.518927125000005, 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.518399877999997, 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.518365895000002, 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.518055302999997, 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.518029169000002, 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.517531262000006, 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.516930172000002, 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:31:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.516381537000001, 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.516173938999998, 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.506558037999994, 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.506547803999993, 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.506537937000004, 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.506527789000003, 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.506057921000007, 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.506035804999996, 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.505748109999999, 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:31:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.320361939999998, 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:08.958 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:08.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:08.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:08.960 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:08.957+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:09.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:31:09.060 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:09.057+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:09.117 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:09.118 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:09.118 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:09.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:leaving image image_client.0-clone41 intact 2024-09-17T23:31:09.173 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:09.171+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:09.186 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:09.186 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:09.206 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:09.206 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:09.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:09.223 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:09.240 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:09.240 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:09.241 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:09.241 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:09.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:09.257 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-17T23:31:09.261 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:09.258+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:09.262 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T23:31:09.263 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T23:31:09.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:09.361 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:09.359+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:09.365 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:09.365 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:09.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:no valid command found; 10 closest matches: 2024-09-17T23:31:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:0 2024-09-17T23:31:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:1 2024-09-17T23:31:09.410 INFO:teuthology.orchestra.run.smithi037.stderr:2 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:abort 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:assert 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:config diff 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:config diff get 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:config get 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:config help [] 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:config set ... 2024-09-17T23:31:09.411 INFO:teuthology.orchestra.run.smithi037.stderr:admin_socket: invalid command 2024-09-17T23:31:09.413 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T23:31:09.413 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-17T23:31:09.462 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:09.460+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:09.488 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:09.489 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:09.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:09.549 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-17T23:31:09.581 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:09.579+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:09.663 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:09.661+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:09.686 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:09.686 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:09.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:09.782 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:09.780+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:09.866 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:09.864+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:09.869 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:09.870 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:09.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:09.887 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:09.885+0000 7f6a30a2f640 -1 osd.5 93 log_to_monitors true 2024-09-17T23:31:09.938 INFO:teuthology.orchestra.run.smithi139.stdout:5794 write 0xa58ba00 thru 0xa58c4db (0xadc bytes) 2024-09-17T23:31:09.939 INFO:teuthology.orchestra.run.smithi139.stdout:5795 write 0x7ab6400 thru 0x7ab7192 (0xd93 bytes) 2024-09-17T23:31:09.939 INFO:teuthology.orchestra.run.smithi139.stdout:5796 write 0x1119600 thru 0x1126a6a (0xd46b bytes) 2024-09-17T23:31:09.940 INFO:teuthology.orchestra.run.smithi139.stdout:5798 punch from 0x582d200 to 0x5837600, (0xa400 bytes) 2024-09-17T23:31:09.966 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:09.964+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:09.978 INFO:teuthology.orchestra.run.smithi139.stdout:5799 punch from 0x2003c00 to 0x2013800, (0xfc00 bytes) 2024-09-17T23:31:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:5800 trunc from 0xba26600 to 0xde3c400 2024-09-17T23:31:10.003 INFO:teuthology.orchestra.run.smithi139.stdout:5803 read 0xcb82200 thru 0xcb90a80 (0xe881 bytes) 2024-09-17T23:31:10.005 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.005 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.010 INFO:teuthology.orchestra.run.smithi139.stdout:5804 read 0xc1eb600 thru 0xc1f8828 (0xd229 bytes) 2024-09-17T23:31:10.011 INFO:teuthology.orchestra.run.smithi139.stdout:5805 read 0xe87800 thru 0xe8eddd (0x75de bytes) 2024-09-17T23:31:10.013 INFO:teuthology.orchestra.run.smithi139.stdout:5806 punch from 0xcc54600 to 0xcc61600, (0xd000 bytes) 2024-09-17T23:31:10.013 INFO:teuthology.orchestra.run.smithi139.stdout:5807 write 0xa04fa00 thru 0xa05e5a7 (0xeba8 bytes) 2024-09-17T23:31:10.013 INFO:teuthology.orchestra.run.smithi139.stdout:5809 punch from 0x2525800 to 0x2532e00, (0xd600 bytes) 2024-09-17T23:31:10.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:10.015 INFO:teuthology.orchestra.run.smithi139.stdout:5810 write 0x8e54200 thru 0x8e58e49 (0x4c4a bytes) 2024-09-17T23:31:10.015 INFO:teuthology.orchestra.run.smithi139.stdout:5812 write 0x3981a00 thru 0x398ddf6 (0xc3f7 bytes) 2024-09-17T23:31:10.015 INFO:teuthology.orchestra.run.smithi139.stdout:5814 read 0x76a2c00 thru 0x76b20eb (0xf4ec bytes) 2024-09-17T23:31:10.015 INFO:teuthology.orchestra.run.smithi139.stdout:5815 write 0x9a6fc00 thru 0x9a7b998 (0xbd99 bytes) 2024-09-17T23:31:10.029 INFO:teuthology.orchestra.run.smithi139.stdout:5818 trunc from 0xde3c400 to 0x1d5fa00 2024-09-17T23:31:10.042 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:31:10.061 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:10.057+0000 7f6a247de640 -1 osd.5 93 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T23:31:10.099 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:10.098+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:10.121 INFO:teuthology.orchestra.run.smithi139.stdout:5819 trunc from 0x1d5fa00 to 0x8c52000 2024-09-17T23:31:10.128 INFO:teuthology.orchestra.run.smithi139.stdout:5821 read 0x5997e00 thru 0x59a5813 (0xda14 bytes) 2024-09-17T23:31:10.129 INFO:teuthology.orchestra.run.smithi139.stdout:5825 read 0x4077800 thru 0x4084e9a (0xd69b bytes) 2024-09-17T23:31:10.138 INFO:teuthology.orchestra.run.smithi139.stdout:5826 trunc from 0x8c52000 to 0xee9c00 2024-09-17T23:31:10.154 INFO:teuthology.orchestra.run.smithi139.stdout:5827 clone 43 order 24 su 131072 sc 10 2024-09-17T23:31:10.168 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:10.167+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:10.186 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.186 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.195 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:10.283 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:10.281+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:10.341 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.342 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:10.393 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:10.392+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:10.457 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.458 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:10.508 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:10.506+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:10.572 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:10.570+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:10.614 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.614 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:10.706 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:10.705+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:10.770 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.771 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.773 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:10.771+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:10.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:10.874 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:10.872+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:10.927 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:10.927 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:10.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:10.979 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:10.977+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:11.075 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:11.074+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:11.089 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:11.089 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:11.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:11.183 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:11.182+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:11.246 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:11.247 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:11.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:11.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:11.296 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:11.294+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:11.379 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:11.377+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:11.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:11.405 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:11.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:11.419 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:11.419 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:11.437 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:11.453 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:11.454 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:11.470 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:11.471 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:11.479 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:11.477+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:11.487 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:11.488 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:11.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:11.505 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-17T23:31:11.524 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:11.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:11.524 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:11.524 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:11.524 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:11.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:11.617 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:11.616+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:11.680 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:11.678+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:11.683 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.644795633000001, 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.644622693000002, 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.644582822999993, 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.350078596000003, 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.102377047000004, 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.092444780999998, 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.079217732000004, 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.538688575999998, 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.538667185999998, 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.538650056999998, 2024-09-17T23:31:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.538632774999996, 2024-09-17T23:31:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.538615746000005, 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.538582872000006, 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.537995166000002, 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.516203024999996, 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:11.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.515665837, 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:11.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.515179720999996, 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:11.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:11.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:11.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.443037119000003, 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:11.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:11.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:11.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:11.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.443015308, 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:11.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:11.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.433572437999999, 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:11.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:11.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:11.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:11.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:11.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:11.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:11.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:11.794 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:11.792+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:11.894 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:11.892+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:11.913 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:11.914 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:11.914 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:11.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:11.995 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:11.994+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:12.030 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:12.030 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:12.030 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:12.031 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:12.031 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:12.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:12.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:31:12.096 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:12.094+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:12.159 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.280174948, 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:12.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:12.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.165 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:12.166 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.26454981800001, 2024-09-17T23:31:12.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:12.166 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.166 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:12.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.264512162, 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:12.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.178 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.264494731, 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:12.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:12.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.2644598, 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.184 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.26442838, 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:12.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:12.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.26441175399999, 2024-09-17T23:31:12.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:12.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:12.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.26439006, 2024-09-17T23:31:12.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:12.204 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 105.264378342, 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.209 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.305239823000001, 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:12.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:12.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.301284658999997, 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:12.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:12.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.222 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.301244001000001, 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.223 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:12.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:12.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.301087266000003, 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:12.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:31:12.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.300984575999998, 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:12.233 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:12.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:12.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.238 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.300948353999999, 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:12.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:12.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.300916059000002, 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:12.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.300761248000001, 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:12.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:12.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:12.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.300474928, 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:12.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:12.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:12.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.300406267, 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:12.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 46.300364293000001, 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:12.264 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:12.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:12.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.269 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:12.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:12.271 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:12.269+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:12.373 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:12.371+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:12.420 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:12.420 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:12.420 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:12.421 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:12.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:12.474 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:12.472+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:12.536 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:12.536 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:12.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:12.544 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:12.586 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:12.584+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:12.675 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:12.673+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:12.695 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:12.696 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.198841787999996, 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:12.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.198821907999999, 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:12.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:12.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.198800370000001, 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.706 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:12.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.710 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196917772000006, 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:12.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:12.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:12.715 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196907655999993, 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:12.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:12.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:12.720 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196895425999998, 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:12.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:12.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.724 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196884857000001, 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.725 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:12.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:12.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:12.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:12.730 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196874969000007, 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:12.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:12.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196864687000001, 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:12.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.196848799999998, 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:12.739 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:12.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:12.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.657923483000005, 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.744 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:12.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.655854097000002, 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:12.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:12.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:12.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.323986728999998, 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.756 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:12.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.839022051000001, 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:12.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:12.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.838989681000001, 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.766 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:12.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.838697304, 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.832873165999999, 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:12.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:12.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:12.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:12.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:12.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:12.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.832831663, 2024-09-17T23:31:12.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:12.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.790 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.790093396000003, 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:12.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.795 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.789991055000002, 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:12.796 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:12.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:12.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:12.801 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:12.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:12.803 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:12.801+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:12.903 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:12.902+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:12.914 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:12.914 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:12.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:13.004 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:13.002+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:31:13.074 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:13.074 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:13.075 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:13.075 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:13.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:13.125 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:13.123+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:13.205 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:13.203+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:13.232 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.387649488999998, 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.378077390000001, 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.374546883999997, 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:13.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.373052178000002, 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:13.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.372869258999998, 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:13.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.372755757999997, 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:13.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:13.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.372731651000002, 2024-09-17T23:31:13.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.372657320000002, 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:13.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:13.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:13.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.372633770999997, 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:13.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:13.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:13.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:13.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.369462570000003, 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:13.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:13.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.365054233999999, 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:13.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:13.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.337782126999997, 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.289 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:13.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:13.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:13.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:13.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:13.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.328599975000003, 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:13.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:13.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:13.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.328568482000001, 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:13.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:13.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.32854279, 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:13.307 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:13.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:13.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.328398237999998, 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:13.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:13.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.326830121, 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:13.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.326632773999997, 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:13.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.326606458999997, 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:13.328 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.326578116, 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:13.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:13.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:13.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:13.340 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:13.340 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:13.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:13.342 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:13.339+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:13.441 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:13.440+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:13.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:13.490 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:13.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:13.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:13.583 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:13.581+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:13.643 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:13.641+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:13.649 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:13.649 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:13.649 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:13.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:13.664 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:13.664 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:13.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:13.684 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:13.702 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:13.702 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:13.720 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:13.721 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:13.739 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:13.740 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:13.743 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:13.742+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:13.758 INFO:teuthology.orchestra.run.smithi139.stdout:truncating image image_client.0-clone42 from 0xee9c00 (overlap 0xee9c00) to 0x0 2024-09-17T23:31:13.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:13.788 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:13.788 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:13.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:13.845 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:13.843+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:13.945 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:13.944+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:13.946 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:13.946 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:13.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:14.046 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:14.045+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.107 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:14.148 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:14.146+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:14.248 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:14.247+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:14.263 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.263 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:14.349 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:14.348+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:14.382 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.382 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.391 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:14.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:14.451 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:14.449+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:14.500 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.500 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:14.543 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.544 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.552 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:14.550+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:14.616 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.617 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:14.652 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:14.651+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.735 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:14.788 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:14.786+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:14.850 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:14.850 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:14.854 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:14.852+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:14.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:14.954 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:14.952+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:15.006 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.006 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:15.056 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:15.054+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:15.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:31:15.112 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-17T23:31:15.156 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:15.154+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:15.166 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.166 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.177 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:15.257 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:15.255+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:15.313 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.313 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:15.391 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:15.361+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:15.458 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:15.457+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:15.492 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.493 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.503 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:15.560 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:15.558+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:15.628 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.628 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:15.681 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:15.680+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:15.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:15.799 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:15.797+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:15.862 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:15.861+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:15.886 INFO:teuthology.orchestra.run.smithi139.stdout:5829 punch from 0x941000 to 0x94e200, (0xd200 bytes) 2024-09-17T23:31:15.902 INFO:teuthology.orchestra.run.smithi139.stdout:5833 read 0x758c00 thru 0x765a23 (0xce24 bytes) 2024-09-17T23:31:15.911 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:15.911 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:15.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:15.924 INFO:teuthology.orchestra.run.smithi139.stdout:5834 write 0x9ad8000 thru 0x9ad8de2 (0xde3 bytes) 2024-09-17T23:31:15.926 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:15.926 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:15.931 INFO:teuthology.orchestra.run.smithi139.stdout:5835 read 0x6dbe400 thru 0x6dc336b (0x4f6c bytes) 2024-09-17T23:31:15.931 INFO:teuthology.orchestra.run.smithi139.stdout:5836 write 0x4ec000 thru 0x4f8552 (0xc553 bytes) 2024-09-17T23:31:15.931 INFO:teuthology.orchestra.run.smithi139.stdout:5838 write 0x7a9f000 thru 0x7aad801 (0xe802 bytes) 2024-09-17T23:31:15.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:15.951 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:15.963 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:15.961+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:15.965 INFO:teuthology.orchestra.run.smithi139.stdout:5839 write 0xb2f7600 thru 0xb300f7e (0x997f bytes) 2024-09-17T23:31:15.972 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:15.972 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:15.985 INFO:teuthology.orchestra.run.smithi139.stdout:5840 write 0xbccb400 thru 0xbccec96 (0x3897 bytes) 2024-09-17T23:31:15.990 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:15.990 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:15.990 INFO:teuthology.orchestra.run.smithi139.stdout:5841 trunc from 0xbccee00 to 0x784b600 2024-09-17T23:31:16.013 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:16.014 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:16.031 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:16.031 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:16.034 INFO:teuthology.orchestra.run.smithi139.stdout:5843 write 0x1d5a400 thru 0x1d6a027 (0xfc28 bytes) 2024-09-17T23:31:16.035 INFO:teuthology.orchestra.run.smithi139.stdout:5847 write 0x2086400 thru 0x2089681 (0x3282 bytes) 2024-09-17T23:31:16.035 INFO:teuthology.orchestra.run.smithi139.stdout:5849 write 0x980a00 thru 0x98dea3 (0xd4a4 bytes) 2024-09-17T23:31:16.036 INFO:teuthology.orchestra.run.smithi139.stdout:5851 read 0x1fe6e00 thru 0x1ff0e72 (0xa073 bytes) 2024-09-17T23:31:16.038 INFO:teuthology.orchestra.run.smithi139.stdout:5853 trunc from 0x784b600 to 0x8b89800 2024-09-17T23:31:16.050 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:16.051 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:16.051 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:16.051 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:16.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:16.064 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:16.062+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:16.069 INFO:teuthology.orchestra.run.smithi139.stdout:5855 write 0xba71800 thru 0xba7b1aa (0x99ab bytes) 2024-09-17T23:31:16.072 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:31:16.081 INFO:teuthology.orchestra.run.smithi139.stdout:5858 write 0x6486c00 thru 0x6491fa2 (0xb3a3 bytes) 2024-09-17T23:31:16.082 INFO:teuthology.orchestra.run.smithi139.stdout:5859 read 0x476e800 thru 0x477af22 (0xc723 bytes) 2024-09-17T23:31:16.094 INFO:teuthology.orchestra.run.smithi139.stdout:5861 trunc from 0xba7b200 to 0x3699c00 2024-09-17T23:31:16.129 INFO:teuthology.orchestra.run.smithi139.stdout:5863 read 0xb78a00 thru 0xb78b33 (0x134 bytes) 2024-09-17T23:31:16.135 INFO:teuthology.orchestra.run.smithi139.stdout:5864 punch from 0xcdd400 to 0xce4200, (0x6e00 bytes) 2024-09-17T23:31:16.153 INFO:teuthology.orchestra.run.smithi139.stdout:5867 write 0x7e1b400 thru 0x7e21229 (0x5e2a bytes) 2024-09-17T23:31:16.160 INFO:teuthology.orchestra.run.smithi139.stdout:5869 read 0x5ef0e00 thru 0x5ef1ca0 (0xea1 bytes) 2024-09-17T23:31:16.160 INFO:teuthology.orchestra.run.smithi139.stdout:5872 read 0x5ddee00 thru 0x5ddf892 (0xa93 bytes) 2024-09-17T23:31:16.161 INFO:teuthology.orchestra.run.smithi139.stdout:5874 read 0x38b6c00 thru 0x38c30d8 (0xc4d9 bytes) 2024-09-17T23:31:16.161 INFO:teuthology.orchestra.run.smithi139.stdout:5876 write 0x34bba00 thru 0x34c2f82 (0x7583 bytes) 2024-09-17T23:31:16.165 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:16.164+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:16.185 INFO:teuthology.orchestra.run.smithi139.stdout:5877 write 0xd71c200 thru 0xd71d8f1 (0x16f2 bytes) 2024-09-17T23:31:16.187 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:16.187 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:16.190 INFO:teuthology.orchestra.run.smithi139.stdout:5878 read 0x5817e00 thru 0x581d173 (0x5374 bytes) 2024-09-17T23:31:16.195 INFO:teuthology.orchestra.run.smithi139.stdout:5879 trunc from 0xd71da00 to 0x954e600 2024-09-17T23:31:16.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:16.226 INFO:teuthology.orchestra.run.smithi139.stdout:5881 trunc from 0x954e600 to 0x612ac00 2024-09-17T23:31:16.250 INFO:teuthology.orchestra.run.smithi139.stdout:5883 write 0x1b89600 thru 0x1b8e8c6 (0x52c7 bytes) 2024-09-17T23:31:16.257 INFO:teuthology.orchestra.run.smithi139.stdout:5884 trunc from 0x612ac00 to 0xa208a00 2024-09-17T23:31:16.266 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:16.264+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:16.274 INFO:teuthology.orchestra.run.smithi139.stdout:5885 trunc from 0xa208a00 to 0x52eae00 2024-09-17T23:31:16.310 INFO:teuthology.orchestra.run.smithi139.stdout:5886 trunc from 0x52eae00 to 0xd99cc00 2024-09-17T23:31:16.322 INFO:teuthology.orchestra.run.smithi139.stdout:5887 trunc from 0xd99cc00 to 0xc8ac000 2024-09-17T23:31:16.331 INFO:teuthology.orchestra.run.smithi139.stdout:5890 write 0x293da00 thru 0x29465ab (0x8bac bytes) 2024-09-17T23:31:16.336 INFO:teuthology.orchestra.run.smithi139.stdout:5891 write 0x8d76400 thru 0x8d7c57f (0x6180 bytes) 2024-09-17T23:31:16.346 INFO:teuthology.orchestra.run.smithi139.stdout:5892 write 0xdb7b000 thru 0xdb8a076 (0xf077 bytes) 2024-09-17T23:31:16.347 INFO:teuthology.orchestra.run.smithi139.stdout:5893 punch from 0x7400 to 0xd000, (0x5c00 bytes) 2024-09-17T23:31:16.347 INFO:teuthology.orchestra.run.smithi139.stdout:5894 write 0x5f87600 thru 0x5f948cb (0xd2cc bytes) 2024-09-17T23:31:16.351 INFO:teuthology.orchestra.run.smithi139.stdout:5895 write 0x7d04800 thru 0x7d0540e (0xc0f bytes) 2024-09-17T23:31:16.353 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.87929991499999, 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.021464101999996, 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.020771808999996, 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.020760228, 2024-09-17T23:31:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.020232981000007, 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.020198997999998, 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.019888406000007, 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.019862271999997, 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.019364365000001, 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.018763274999998, 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.018214639999997, 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.018007041999994, 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.008391141000004, 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.008380907000003, 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:16.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:16.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.00837104, 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:16.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:16.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.008360891999999, 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:16.430 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:16.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:16.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.007891024000003, 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:16.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:16.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.007868908000006, 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:16.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:16.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.007581212999995, 2024-09-17T23:31:16.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:31:16.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:31:16.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.822195042999994, 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:16.450 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:16.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:16.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:16.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.457 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:16.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:16.459 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:16.456+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5896 read 0x167ac00 thru 0x167c2b3 (0x16b4 bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5898 read 0x576e400 thru 0x5775d63 (0x7964 bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5899 write 0x327f200 thru 0x3288c56 (0x9a57 bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5900 read 0xbd03200 thru 0xbd0981d (0x661e bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5901 read 0x48ebe00 thru 0x48f39fb (0x7bfc bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5902 read 0xb80a00 thru 0xb8cde6 (0xc3e7 bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5903 write 0x978d800 thru 0x9791814 (0x4015 bytes) 2024-09-17T23:31:16.462 INFO:teuthology.orchestra.run.smithi139.stdout:5904 punch from 0x84e7c00 to 0x84e9400, (0x1800 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5905 write 0xa107200 thru 0xa111e36 (0xac37 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5907 read 0x4fb3400 thru 0x4fb7a10 (0x4611 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5908 read 0x9879000 thru 0x9881186 (0x8187 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5911 write 0xab8d000 thru 0xab94cb7 (0x7cb8 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5912 write 0x9963400 thru 0x9973091 (0xfc92 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5913 read 0x6049600 thru 0x604df60 (0x4961 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5914 punch from 0x75e0a00 to 0x75e4c00, (0x4200 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5916 write 0xb5c9c00 thru 0xb5cf805 (0x5c06 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5919 write 0xb7be200 thru 0xb7c473f (0x6540 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5920 write 0x1f35000 thru 0x1f3ad0c (0x5d0d bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5922 write 0xae73a00 thru 0xae80270 (0xc871 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5923 read 0x7798400 thru 0x77a41a7 (0xbda8 bytes) 2024-09-17T23:31:16.463 INFO:teuthology.orchestra.run.smithi139.stdout:5924 read 0x95fce00 thru 0x9602304 (0x5505 bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5925 trunc from 0xdb8a200 to 0x8d55800 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5926 write 0x36b5800 thru 0x36b7bba (0x23bb bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5929 punch from 0xfc7a00 to 0xfca000, (0x2600 bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5931 read 0x63fd000 thru 0x6404511 (0x7512 bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5933 read 0x3ce8c00 thru 0x3ceee6f (0x6270 bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5934 write 0xea85e00 thru 0xea929e2 (0xcbe3 bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5936 write 0x8a95400 thru 0x8a9c57c (0x717d bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5937 read 0xba7e400 thru 0xba7f9d0 (0x15d1 bytes) 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5938 trunc from 0xea92a00 to 0x7d90000 2024-09-17T23:31:16.464 INFO:teuthology.orchestra.run.smithi139.stdout:5939 read 0x6842c00 thru 0x6845c0b (0x300c bytes) 2024-09-17T23:31:16.470 INFO:teuthology.orchestra.run.smithi139.stdout:5942 write 0xa3a5800 thru 0xa3b1300 (0xbb01 bytes) 2024-09-17T23:31:16.472 INFO:teuthology.orchestra.run.smithi139.stdout:5943 write 0x2fa6e00 thru 0x2fab31c (0x451d bytes) 2024-09-17T23:31:16.472 INFO:teuthology.orchestra.run.smithi139.stdout:5946 read 0x5b33000 thru 0x5b33b29 (0xb2a bytes) 2024-09-17T23:31:16.477 INFO:teuthology.orchestra.run.smithi139.stdout:5947 punch from 0x544a000 to 0x5459400, (0xf400 bytes) 2024-09-17T23:31:16.482 INFO:teuthology.orchestra.run.smithi139.stdout:5948 trunc from 0xa3b1400 to 0x2ee8200 2024-09-17T23:31:16.512 INFO:teuthology.orchestra.run.smithi139.stdout:5950 trunc from 0x2ee8200 to 0x9347800 2024-09-17T23:31:16.521 INFO:teuthology.orchestra.run.smithi139.stdout:5951 read 0x1018c00 thru 0x101d291 (0x4692 bytes) 2024-09-17T23:31:16.523 INFO:teuthology.orchestra.run.smithi139.stdout:5952 read 0x2371000 thru 0x237709e (0x609f bytes) 2024-09-17T23:31:16.523 INFO:teuthology.orchestra.run.smithi139.stdout:5953 write 0x4247600 thru 0x42484c5 (0xec6 bytes) 2024-09-17T23:31:16.526 INFO:teuthology.orchestra.run.smithi139.stdout:5954 write 0x8e7da00 thru 0x8e8906a (0xb66b bytes) 2024-09-17T23:31:16.526 INFO:teuthology.orchestra.run.smithi139.stdout:5956 read 0x8a28a00 thru 0x8a2eb4d (0x614e bytes) 2024-09-17T23:31:16.526 INFO:teuthology.orchestra.run.smithi139.stdout:5958 read 0x333ce00 thru 0x3342fed (0x61ee bytes) 2024-09-17T23:31:16.526 INFO:teuthology.orchestra.run.smithi139.stdout:5962 write 0x7002a00 thru 0x701253c (0xfb3d bytes) 2024-09-17T23:31:16.540 INFO:teuthology.orchestra.run.smithi139.stdout:5965 write 0xa9e6c00 thru 0xa9eb657 (0x4a58 bytes) 2024-09-17T23:31:16.549 INFO:teuthology.orchestra.run.smithi139.stdout:5966 trunc from 0xa9eb800 to 0x8a7d800 2024-09-17T23:31:16.558 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:16.557+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:16.573 INFO:teuthology.orchestra.run.smithi139.stdout:5967 trunc from 0x8a7d800 to 0x9a11a00 2024-09-17T23:31:16.605 INFO:teuthology.orchestra.run.smithi139.stdout:5969 write 0x9dd7000 thru 0x9de1164 (0xa165 bytes) 2024-09-17T23:31:16.611 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:16.612 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:16.612 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:16.612 INFO:teuthology.orchestra.run.smithi139.stdout:5971 punch from 0x48a5c00 to 0x48b3200, (0xd600 bytes) 2024-09-17T23:31:16.613 INFO:teuthology.orchestra.run.smithi139.stdout:5974 punch from 0x2f5c200 to 0x2f5d600, (0x1400 bytes) 2024-09-17T23:31:16.613 INFO:teuthology.orchestra.run.smithi139.stdout:5975 read 0x20e5800 thru 0x20ec174 (0x6975 bytes) 2024-09-17T23:31:16.614 INFO:teuthology.orchestra.run.smithi139.stdout:5977 read 0x31cea00 thru 0x31d1e47 (0x3448 bytes) 2024-09-17T23:31:16.615 INFO:teuthology.orchestra.run.smithi139.stdout:5979 punch from 0x6842c00 to 0x6851000, (0xe400 bytes) 2024-09-17T23:31:16.615 INFO:teuthology.orchestra.run.smithi139.stdout:5980 read 0x88b6a00 thru 0x88c3452 (0xca53 bytes) 2024-09-17T23:31:16.617 INFO:teuthology.orchestra.run.smithi139.stdout:5981 read 0x3de0800 thru 0x3de77c0 (0x6fc1 bytes) 2024-09-17T23:31:16.617 INFO:teuthology.orchestra.run.smithi139.stdout:5982 read 0x4f6b400 thru 0x4f764ab (0xb0ac bytes) 2024-09-17T23:31:16.618 INFO:teuthology.orchestra.run.smithi139.stdout:5984 read 0x3431800 thru 0x3432558 (0xd59 bytes) 2024-09-17T23:31:16.618 INFO:teuthology.orchestra.run.smithi139.stdout:5985 punch from 0x9d6000 to 0x9e0a00, (0xaa00 bytes) 2024-09-17T23:31:16.618 INFO:teuthology.orchestra.run.smithi139.stdout:5987 read 0x7088e00 thru 0x708ea7f (0x5c80 bytes) 2024-09-17T23:31:16.619 INFO:teuthology.orchestra.run.smithi139.stdout:5988 read 0x3f31c00 thru 0x3f3c4fb (0xa8fc bytes) 2024-09-17T23:31:16.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:16.629 INFO:teuthology.orchestra.run.smithi139.stdout:5990 write 0xaedb800 thru 0xaeeb016 (0xf817 bytes) 2024-09-17T23:31:16.636 INFO:teuthology.orchestra.run.smithi139.stdout:5991 write 0x8c2b600 thru 0x8c3149b (0x5e9c bytes) 2024-09-17T23:31:16.637 INFO:teuthology.orchestra.run.smithi139.stdout:5992 read 0x1d04400 thru 0x1d0e39a (0x9f9b bytes) 2024-09-17T23:31:16.637 INFO:teuthology.orchestra.run.smithi139.stdout:5993 punch from 0xc48a00 to 0xc4a200, (0x1800 bytes) 2024-09-17T23:31:16.638 INFO:teuthology.orchestra.run.smithi139.stdout:5995 write 0x13cd400 thru 0x13d7f97 (0xab98 bytes) 2024-09-17T23:31:16.639 INFO:teuthology.orchestra.run.smithi139.stdout:5996 read 0x9928200 thru 0x99354a0 (0xd2a1 bytes) 2024-09-17T23:31:16.660 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:16.658+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:16.761 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:16.759+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:16.770 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:16.770 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:16.770 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:16.770 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:16.770 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:16.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:16.815 INFO:teuthology.orchestra.run.smithi139.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-17T23:31:16.862 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:16.860+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:16.889 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:16.889 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:16.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.850850402999995, 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:16.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:16.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:16.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.850677462999997, 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:16.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:16.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.850637593000002, 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:16.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:16.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.556133365999997, 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:16.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.308431816999999, 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:16.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.298499551000006, 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:16.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.285272501999998, 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744743346000007, 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744721956000006, 2024-09-17T23:31:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744704827000007, 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744687545000005, 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744670515999999, 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744637642000001, 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 76.744049935999996, 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.722257795000004, 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.721720606999995, 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.721234491000004, 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.649091888999997, 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.649070078000001, 2024-09-17T23:31:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.639627208, 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:16.997 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:16.998 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:16.999 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:16.996+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:17.099 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:17.097+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:17.114 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:17.114 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:17.114 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:17.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:17.199 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:17.198+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:17.273 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:17.273 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:17.274 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:17.274 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:17.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:17.327 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:17.325+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:17.391 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.513144251, 2024-09-17T23:31:17.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:17.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:17.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:17.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497519121, 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.400 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.49748146500001, 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.407 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:17.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.413 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497464034, 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.414 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:17.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497429103, 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.421 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:17.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:17.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.427 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497397683, 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:17.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497381057, 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.434 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497359363, 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.440 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:17.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.497347645, 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:17.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.450 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.538209125999998, 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:17.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:17.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:17.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:17.458 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.534253962000001, 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:17.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.463 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.534213303999998, 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.464 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:17.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:17.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:17.469 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:522 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.534056569000001, 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000968028, 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.470 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 522 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853834+0000", 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853840+0000", 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4199999999999998e-06 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2130000000000002e-06 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6720000000000001e-06 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854069+0000", 2024-09-17T23:31:17.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00021588799999999999 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854214+0000", 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014555899999999999 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854424+0000", 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0002101400000001 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854663+0000", 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000239044 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854802+0000", 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013866700000000001 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.475 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.533953879000002, 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.476 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:17.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.481 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.533917657000003, 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:17.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:17.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.487 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.533885361999999, 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.488 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:17.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:17.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.493 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.533730550999998, 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:17.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:17.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:17.502 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.533444230999997, 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:17.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.533375569999997, 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.509 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:17.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:17.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 51.533333595999999, 2024-09-17T23:31:17.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:17.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:17.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:17.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.520 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:17.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:17.522 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:17.520+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:17.623 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:17.621+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:17.670 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:17.670 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:17.670 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:17.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:17.724 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:17.722+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:17.785 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:17.785 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:17.785 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:17.793 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:17.825 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:17.823+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:17.902 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:17.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.40542416700001, 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:17.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:17.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:17.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.908 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.909 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.405404287, 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.910 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.405382749, 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:17.914 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:17.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:17.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.403500151, 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.923 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.403490035, 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:17.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:17.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.929 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.40347780499999, 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:17.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:17.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.403467236, 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.934 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:17.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.938 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.403457348, 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:17.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.403447066, 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.947 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 100.40343117899999, 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.948 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:17.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:17.952 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.864505862000001, 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:17.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.862436475999999, 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:17.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:17.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 78.530569107999995, 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.965 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 57.045604429999997, 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:17.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:17.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.974 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 57.045572059999998, 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:17.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:17.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 57.045279682999997, 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:17.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:17.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.985 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 57.039455545000003, 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:17.990 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 57.039414041999997, 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.991 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:17.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:17.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:17.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:17.996 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.996675775, 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:17.997 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:17.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:18.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.001 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 56.996573433999998, 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:18.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:18.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:18.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.007 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:18.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:18.009 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:18.006+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:18.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:18.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:31:18.108 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:18.107+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:18.167 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:18.168 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:18.168 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:18.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:18.210 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:18.208+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:18.310 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:18.309+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:18.326 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:18.326 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:18.326 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:18.334 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:18.387 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:18.387 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:18.404 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:18.405 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:18.411 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:18.409+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:18.420 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:18.438 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:18.445 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.445 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.1f 108/107 [PushOp(3:fbf9aaa9:::rbd_data.1152abcd6d35.0000000000000000:head, version: 101'368, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fbf9aaa9:::rbd_data.1152abcd6d35.0000000000000000:head@101'368, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.291050426, 2024-09-17T23:31:18.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021473270000000001, 2024-09-17T23:31:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4600000000000004e-07 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151567+0000", 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00085061000000000002 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151590+0000", 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2694e-05 2024-09-17T23:31:18.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151609+0000", 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.893e-05 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151686+0000", 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7415000000000003e-05 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151701+0000", 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5299e-05 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.152863+0000", 2024-09-17T23:31:18.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011617330000000001 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.1f 108/107 [PushOp(3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:2a, version: 103'372, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:2a@103'372, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:18.452 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.2796984819999997, 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020526070000000001, 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162843+0000", 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00077470399999999997 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162862+0000", 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9332000000000001e-05 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162879+0000", 2024-09-17T23:31:18.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.632e-05 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162952+0000", 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2787000000000006e-05 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162960+0000", 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8820000000000007e-06 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.164120+0000", 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011598769999999999 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.11 108/107 [PushOp(3:88754e9c:::rbd_data.1152c10d4b9c.0000000000000000:2a, version: 103'314, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:88754e9c:::rbd_data.1152c10d4b9c.0000000000000000:2a@103'314, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.168151+0000", 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.2736152169999997, 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020437020000000001, 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168151+0000", 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168151+0000", 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168152+0000", 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.75e-07 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168963+0000", 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00081175799999999999 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168980+0000", 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6646e-05 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168994+0000", 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4005999999999999e-05 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.169087+0000", 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2953000000000003e-05 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.169101+0000", 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3872e-05 2024-09-17T23:31:18.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.170195+0000", 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010937919999999999 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.a 108/107 [PushOp(3:536e2895:::rbd_data.1152c10d4b9c.000000000000000c:2a, version: 103'497, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:536e2895:::rbd_data.1152c10d4b9c.000000000000000c:2a@103'497, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 6.1835439689999996, 2024-09-17T23:31:18.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002379188, 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258223+0000", 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.82e-07 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259145+0000", 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00092170299999999995 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259163+0000", 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8284e-05 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259176+0000", 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2747e-05 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259259+0000", 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2981999999999999e-05 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259271+0000", 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2711000000000001e-05 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.260601+0000", 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013301789999999999 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:20738 3.1c e109/108 3:3831950c:::rbd_directory:head v 109'350, mlcod=0'0)", 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.727861796, 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026884439999999999, 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713905+0000", 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7599999999999997e-07 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713910+0000", 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3380000000000004e-06 2024-09-17T23:31:18.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713922+0000", 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1185999999999999e-05 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713927+0000", 2024-09-17T23:31:18.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.524e-06 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713963+0000", 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6369999999999999e-05 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713981+0000", 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7986e-05 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.716542+0000", 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025610540000000001 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.716593+0000", 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.041e-05 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1011 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 5636096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.471 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.3286645670000001, 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025798650000000002, 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1011 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113103+0000", 2024-09-17T23:31:18.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.708e-06 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113110+0000", 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0160000000000003e-06 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113127+0000", 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7019000000000001e-05 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113135+0000", 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6559999999999992e-06 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113185+0000", 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0204999999999998e-05 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113189+0000", 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9249999999999997e-06 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114714+0000", 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0015248060000000001 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114759+0000", 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5420000000000002e-05 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114842+0000", 2024-09-17T23:31:18.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2470000000000002e-05 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115485+0000", 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00064293800000000002 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115630+0000", 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000145301 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115659+0000", 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9251000000000002e-05 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115682+0000", 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-05 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1209 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 15597568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.478 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.2370381149999998, 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044407630000000003, 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1209 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204729+0000", 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.077e-06 2024-09-17T23:31:18.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204736+0000", 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7889999999999997e-06 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204747+0000", 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0793e-05 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204754+0000", 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3379999999999999e-06 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204820+0000", 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6235000000000005e-05 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204911+0000", 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1024000000000001e-05 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.205182+0000", 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000270134 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.208700+0000", 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035188810000000002 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209111+0000", 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00041066099999999999 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209144+0000", 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2653000000000001e-05 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209169+0000", 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5177999999999998e-05 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1629 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.0623143129999999, 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038611639999999998, 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1629 2024-09-17T23:31:18.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.486 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379453+0000", 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1429999999999999e-06 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379457+0000", 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0969999999999999e-06 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379469+0000", 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1868999999999999e-05 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379476+0000", 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2400000000000001e-06 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379533+0000", 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6666999999999998e-05 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379584+0000", 2024-09-17T23:31:18.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1057000000000003e-05 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379647+0000", 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2778000000000001e-05 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380452+0000", 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080491199999999997 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383214+0000", 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002761922 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383272+0000", 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8184999999999999e-05 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383313+0000", 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1294000000000001e-05 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.490 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1624 3.11 e112/107 3:8fa7046a:::rbd_data.1152d7c8a97e.000000000000000a:head v 112'327, mlcod=112'325)", 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.0617435359999998, 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031108070000000002, 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7000000000000004e-07 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380030+0000", 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.455e-06 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380041+0000", 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0786e-05 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380048+0000", 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380105+0000", 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6938000000000003e-05 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380124+0000", 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9332000000000001e-05 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383100+0000", 2024-09-17T23:31:18.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002975924 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383133+0000", 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3688e-05 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:18.495 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.0616989019999998, 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:18.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:18.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1643 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14811136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 2.054064763, 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042815140000000002, 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1643 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387703+0000", 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.308e-06 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387708+0000", 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.865e-06 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387723+0000", 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.556e-05 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387732+0000", 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5099999999999998e-06 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387795+0000", 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2929999999999995e-05 2024-09-17T23:31:18.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387897+0000", 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102815 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388033+0000", 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013546899999999999 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388921+0000", 2024-09-17T23:31:18.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088850099999999998 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391906+0000", 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002984202 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391944+0000", 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8565000000000001e-05 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391983+0000", 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8788999999999997e-05 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:2058 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14538752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.562425+0000", 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.879340789, 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020832559999999999, 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2058 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562425+0000", 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562425+0000", 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562427+0000", 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.091e-06 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562432+0000", 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9020000000000004e-06 2024-09-17T23:31:18.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562449+0000", 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6375000000000002e-05 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562457+0000", 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562523+0000", 2024-09-17T23:31:18.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5550999999999996e-05 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562614+0000", 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1439000000000003e-05 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562749+0000", 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134685 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564083+0000", 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001333905 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564440+0000", 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00035695600000000002 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564477+0000", 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7054999999999997e-05 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564509+0000", 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1847999999999997e-05 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:2237 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1554432~18432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.802657942, 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021808159999999999, 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2237 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639110+0000", 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6640000000000001e-06 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639124+0000", 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4316e-05 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639135+0000", 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1134e-05 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639141+0000", 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8379999999999998e-06 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639200+0000", 2024-09-17T23:31:18.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9005999999999997e-05 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639286+0000", 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6138999999999996e-05 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639431+0000", 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014437799999999999 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641204+0000", 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017734669999999999 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641230+0000", 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5428000000000001e-05 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641263+0000", 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3337000000000003e-05 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641289+0000", 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6109e-05 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21422 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.4610164459999999, 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044109429999999996, 2024-09-17T23:31:18.518 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21422 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980751+0000", 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980755+0000", 2024-09-17T23:31:18.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980768+0000", 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2862e-05 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980776+0000", 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8359999999999992e-06 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980833+0000", 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6749000000000002e-05 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980879+0000", 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6050999999999997e-05 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980977+0000", 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8488999999999998e-05 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982120+0000", 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001143257 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985096+0000", 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00297569 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985133+0000", 2024-09-17T23:31:18.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7134000000000001e-05 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985161+0000", 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7514000000000001e-05 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:18.524 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.4594478500000001, 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:18.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:18.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:18.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:18.530 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.35881521300000002, 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:18.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:18.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21654 3.0 3:02844c61:::rbd_header.11525992f601:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.094887+0000", 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.34687891500000001, 2024-09-17T23:31:18.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0019996609999999998, 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21654 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094887+0000", 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094887+0000", 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094889+0000", 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.291e-06 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094894+0000", 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2809999999999999e-06 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094909+0000", 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4620000000000001e-05 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094917+0000", 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8019999999999998e-06 2024-09-17T23:31:18.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.095007+0000", 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0113999999999998e-05 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.095088+0000", 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0765000000000003e-05 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.095257+0000", 2024-09-17T23:31:18.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00016835799999999999 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096189+0000", 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00093219500000000003 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096773+0000", 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00058436399999999998 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096841+0000", 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8167e-05 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096887+0000", 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5704000000000002e-05 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21708 3.0 3:02844c61:::rbd_header.11525992f601:head [watch unwatch cookie 140329533919712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:18.543 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.299139445, 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020509819999999998, 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21708 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:18.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.89e-07 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142637+0000", 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3160000000000003e-06 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142644+0000", 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4429999999999997e-06 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142649+0000", 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142895+0000", 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00024679699999999999 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142943+0000", 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7552000000000001e-05 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.143062+0000", 2024-09-17T23:31:18.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011877799999999999 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144497+0000", 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014351819999999999 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144516+0000", 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9267e-05 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144575+0000", 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9032e-05 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144678+0000", 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102526 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:21741 3.a e114/107 3:549a6936:::rbd_data.11525992f601.0000000000000002:head v 114'526, mlcod=112'524)", 2024-09-17T23:31:18.549 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.26873791200000002, 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031310919999999998, 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173029+0000", 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8899999999999998e-07 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173037+0000", 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3350000000000001e-06 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173048+0000", 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0995000000000001e-05 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173054+0000", 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5239999999999997e-06 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173133+0000", 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9077000000000002e-05 2024-09-17T23:31:18.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173155+0000", 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1630000000000001e-05 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176110+0000", 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029556130000000002 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176159+0000", 2024-09-17T23:31:18.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8928999999999999e-05 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21759 3.17 3:ee28c85f:::rbd_data.11525992f601.00000000000000ad:head [delete] snapc 2c=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.26242511099999999, 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025726450000000001, 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:18.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21759 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:18.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179342+0000", 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.164e-06 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179347+0000", 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0050000000000004e-06 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179364+0000", 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7059e-05 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179373+0000", 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6580000000000008e-06 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179456+0000", 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2899999999999996e-05 2024-09-17T23:31:18.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179541+0000", 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4640000000000003e-05 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179660+0000", 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000119169 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181588+0000", 2024-09-17T23:31:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0019279729999999999 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181773+0000", 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001852 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181843+0000", 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0487000000000005e-05 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:18.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181914+0000", 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0389999999999995e-05 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:18.560 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:18.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:18.562 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:18.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:18.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:18.563 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:18.564 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:18.560+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:18.662 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:18.660+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:18.712 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:18.712 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:18.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:18.764 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:18.763+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:18.828 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:18.828 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:18.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:18.864 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:18.862+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:18.966 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:18.964+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:18.983 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:18.983 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:18.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:19.067 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:19.065+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:19.085 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:31:19.103 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:19.103 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:19.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:19.197 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:19.195+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:19.261 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:19.261 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:19.268 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:19.266+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:19.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:19.369 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:19.368+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:19.417 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:19.417 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:19.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:19.512 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:19.510+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:19.553 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:31:19.553 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:19.554 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-17T23:31:19.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-17T23:31:19.571 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:19.569+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:19.574 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:19.574 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:19.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:19.672 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:19.671+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:19.733 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:19.733 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:19.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:19.774 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:19.772+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:19.857 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:31:19.856+0000 7f21c94c5640 -1 mon.a@0(leader).osd e115 definitely_dead 0 2024-09-17T23:31:19.875 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:19.873+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:19.890 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:19.891 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:19.899 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:19.975 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:19.974+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:20.019 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:20.019 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:20.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:20.077 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:20.075+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.1. 2024-09-17T23:31:20.104 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-17T23:31:20.142 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:20.142 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:20.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:20.192 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:20.190+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:20.278 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:20.277+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:20.299 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:20.299 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:20.307 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:20.392 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:20.390+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:20.456 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:20.456 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:20.465 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:20.481 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:20.479+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:20.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:20.582 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:20.580+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:20.612 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:20.613 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:20.621 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:20.693 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:20.693 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:20.705 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:20.703+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:20.710 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:20.710 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:20.726 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:20.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:20.743 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:20.760 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:20.760 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:20.771 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.274381401, 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:20.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.274361521, 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.776 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:20.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.783 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.274339983, 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.787 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.272457385, 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.788 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:20.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:20.792 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.272447269, 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:20.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:20.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.272435039, 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.801 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.27242447, 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:20.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.272414582, 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.806 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:20.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:20.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:20.810 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.27240430000001, 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:20.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.812 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:20.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 103.272388413, 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:20.815 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.819 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 82.733463095999994, 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:20.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:20.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.825 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 82.731393710000006, 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:20.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:20.831 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 81.399526342000001, 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:20.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.914561663999997, 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:20.836 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:20.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:20.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:20.841 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.914529293999998, 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:20.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:20.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.844 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.846 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.914236916999997, 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:20.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:20.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.908412779000003, 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:20.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.908371275999997, 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.857 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:20.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:20.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.865633009, 2024-09-17T23:31:20.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:20.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.865 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:20.867 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 59.865530667999998, 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:20.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:20.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.872 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:20.873 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:20.873 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:20.874 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:20.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:20.875 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:20.872+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:20.876 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:20.975 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:20.973+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:21.023 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:21.023 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:21.023 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:21.023 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:21.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:21.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:31:21.117 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:21.115+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:21.177 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:21.174+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:21.179 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:21.179 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:21.179 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:21.179 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:21.180 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:21.189 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:21.277 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:21.275+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.1f 108/107 [PushOp(3:fbf9aaa9:::rbd_data.1152abcd6d35.0000000000000000:head, version: 101'368, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fbf9aaa9:::rbd_data.1152abcd6d35.0000000000000000:head@101'368, size: 6291456, copy_subset: [(0, 6291456)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:21.338 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.184402424, 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021473270000000001, 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.150716+0000", 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4600000000000004e-07 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151567+0000", 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00085061000000000002 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151590+0000", 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2694e-05 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151609+0000", 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.893e-05 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151686+0000", 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7415000000000003e-05 2024-09-17T23:31:21.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.151701+0000", 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5299e-05 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.152863+0000", 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011617330000000001 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.1f 108/107 [PushOp(3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:2a, version: 103'372, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:2a@103'372, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1730504800000006, 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020526070000000001, 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162068+0000", 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162843+0000", 2024-09-17T23:31:21.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00077470399999999997 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162862+0000", 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9332000000000001e-05 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162879+0000", 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.632e-05 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162952+0000", 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2787000000000006e-05 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.162960+0000", 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8820000000000007e-06 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.164120+0000", 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011598769999999999 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.346 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.11 108/107 [PushOp(3:88754e9c:::rbd_data.1152c10d4b9c.0000000000000000:2a, version: 103'314, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:88754e9c:::rbd_data.1152c10d4b9c.0000000000000000:2a@103'314, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.168151+0000", 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1669672149999997, 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020437020000000001, 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168151+0000", 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168151+0000", 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168152+0000", 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.75e-07 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168963+0000", 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00081175799999999999 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168980+0000", 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6646e-05 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.168994+0000", 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4005999999999999e-05 2024-09-17T23:31:21.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.169087+0000", 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2953000000000003e-05 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.169101+0000", 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3872e-05 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.170195+0000", 2024-09-17T23:31:21.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010937919999999999 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.a 108/107 [PushOp(3:536e2895:::rbd_data.1152c10d4b9c.000000000000000c:2a, version: 103'497, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:536e2895:::rbd_data.1152c10d4b9c.000000000000000c:2a@103'497, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.0768959670000005, 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002379188, 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258223+0000", 2024-09-17T23:31:21.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.82e-07 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259145+0000", 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00092170299999999995 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259163+0000", 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8284e-05 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259176+0000", 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2747e-05 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259259+0000", 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2981999999999999e-05 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259271+0000", 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2711000000000001e-05 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.260601+0000", 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013301789999999999 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:20738 3.1c e109/108 3:3831950c:::rbd_directory:head v 109'350, mlcod=0'0)", 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.621213794, 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026884439999999999, 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713905+0000", 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7599999999999997e-07 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713910+0000", 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3380000000000004e-06 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713922+0000", 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1185999999999999e-05 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713927+0000", 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.524e-06 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713963+0000", 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6369999999999999e-05 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713981+0000", 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7986e-05 2024-09-17T23:31:21.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.716542+0000", 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025610540000000001 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.716593+0000", 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.041e-05 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1011 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 5636096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.2220165649999997, 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025798650000000002, 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1011 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113103+0000", 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.708e-06 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113110+0000", 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0160000000000003e-06 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113127+0000", 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7019000000000001e-05 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113135+0000", 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6559999999999992e-06 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113185+0000", 2024-09-17T23:31:21.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0204999999999998e-05 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113189+0000", 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9249999999999997e-06 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114714+0000", 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0015248060000000001 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114759+0000", 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5420000000000002e-05 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114842+0000", 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2470000000000002e-05 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115485+0000", 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00064293800000000002 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115630+0000", 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000145301 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115659+0000", 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9251000000000002e-05 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115682+0000", 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-05 2024-09-17T23:31:21.366 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1209 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 15597568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.1303901129999998, 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044407630000000003, 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1209 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204729+0000", 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.077e-06 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204736+0000", 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7889999999999997e-06 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204747+0000", 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0793e-05 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204754+0000", 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3379999999999999e-06 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204820+0000", 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6235000000000005e-05 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204911+0000", 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1024000000000001e-05 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.205182+0000", 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000270134 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.208700+0000", 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035188810000000002 2024-09-17T23:31:21.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209111+0000", 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00041066099999999999 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209144+0000", 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2653000000000001e-05 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209169+0000", 2024-09-17T23:31:21.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5177999999999998e-05 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1629 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.9556663109999999, 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038611639999999998, 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.373 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1629 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:21.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379453+0000", 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1429999999999999e-06 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379457+0000", 2024-09-17T23:31:21.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0969999999999999e-06 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379469+0000", 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1868999999999999e-05 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379476+0000", 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2400000000000001e-06 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379533+0000", 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6666999999999998e-05 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379584+0000", 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1057000000000003e-05 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379647+0000", 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2778000000000001e-05 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380452+0000", 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080491199999999997 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383214+0000", 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002761922 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383272+0000", 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8184999999999999e-05 2024-09-17T23:31:21.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383313+0000", 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1294000000000001e-05 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1624 3.11 e112/107 3:8fa7046a:::rbd_data.1152d7c8a97e.000000000000000a:head v 112'327, mlcod=112'325)", 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.9550955339999998, 2024-09-17T23:31:21.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031108070000000002, 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7000000000000004e-07 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380030+0000", 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.455e-06 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380041+0000", 2024-09-17T23:31:21.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0786e-05 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380048+0000", 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380105+0000", 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6938000000000003e-05 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380124+0000", 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9332000000000001e-05 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383100+0000", 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002975924 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383133+0000", 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3688e-05 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.9550508999999998, 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:21.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:21.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1643 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14811136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.9474167610000004, 2024-09-17T23:31:21.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042815140000000002, 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1643 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:21.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387703+0000", 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.308e-06 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387708+0000", 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.865e-06 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387723+0000", 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.556e-05 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387732+0000", 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5099999999999998e-06 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387795+0000", 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2929999999999995e-05 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387897+0000", 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102815 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388033+0000", 2024-09-17T23:31:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013546899999999999 2024-09-17T23:31:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388921+0000", 2024-09-17T23:31:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088850099999999998 2024-09-17T23:31:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391906+0000", 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002984202 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391944+0000", 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8565000000000001e-05 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391983+0000", 2024-09-17T23:31:21.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8788999999999997e-05 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:2058 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14538752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.562425+0000", 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.7726927870000004, 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020832559999999999, 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.398 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2058 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562425+0000", 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562425+0000", 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562427+0000", 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.091e-06 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562432+0000", 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9020000000000004e-06 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562449+0000", 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6375000000000002e-05 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562457+0000", 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562523+0000", 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5550999999999996e-05 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.401 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562614+0000", 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1439000000000003e-05 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.562749+0000", 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134685 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564083+0000", 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001333905 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564440+0000", 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00035695600000000002 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564477+0000", 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7054999999999997e-05 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.564509+0000", 2024-09-17T23:31:21.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1847999999999997e-05 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:2237 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1554432~18432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.6960099399999997, 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021808159999999999, 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.404 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2237 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:21.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639110+0000", 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6640000000000001e-06 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639124+0000", 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4316e-05 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.406 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639135+0000", 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1134e-05 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639141+0000", 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8379999999999998e-06 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639200+0000", 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9005999999999997e-05 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639286+0000", 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6138999999999996e-05 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639431+0000", 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014437799999999999 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641204+0000", 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017734669999999999 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641230+0000", 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5428000000000001e-05 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641263+0000", 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3337000000000003e-05 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641289+0000", 2024-09-17T23:31:21.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6109e-05 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21422 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.3543684440000003, 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044109429999999996, 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.410 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21422 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:21.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980751+0000", 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980755+0000", 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980768+0000", 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2862e-05 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980776+0000", 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8359999999999992e-06 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980833+0000", 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6749000000000002e-05 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980879+0000", 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6050999999999997e-05 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980977+0000", 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8488999999999998e-05 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982120+0000", 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001143257 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985096+0000", 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00297569 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985133+0000", 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7134000000000001e-05 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985161+0000", 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7514000000000001e-05 2024-09-17T23:31:21.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.3527998480000001, 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.416 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:21.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.421 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.2521672110000002, 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:21.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:21.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:21.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21654 3.0 3:02844c61:::rbd_header.11525992f601:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:21.428 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.094887+0000", 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.240230913, 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0019996609999999998, 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21654 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094887+0000", 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094887+0000", 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094889+0000", 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.291e-06 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094894+0000", 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2809999999999999e-06 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094909+0000", 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4620000000000001e-05 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.094917+0000", 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8019999999999998e-06 2024-09-17T23:31:21.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.095007+0000", 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0113999999999998e-05 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.095088+0000", 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0765000000000003e-05 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.095257+0000", 2024-09-17T23:31:21.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00016835799999999999 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096189+0000", 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00093219500000000003 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096773+0000", 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00058436399999999998 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096841+0000", 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8167e-05 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.096887+0000", 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5704000000000002e-05 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21708 3.0 3:02844c61:::rbd_header.11525992f601:head [watch unwatch cookie 140329533919712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:21.434 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.1924914430000002, 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0020509819999999998, 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21708 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142627+0000", 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.89e-07 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142637+0000", 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3160000000000003e-06 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142644+0000", 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4429999999999997e-06 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142649+0000", 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:21.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142895+0000", 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00024679699999999999 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.142943+0000", 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7552000000000001e-05 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:21.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.143062+0000", 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011877799999999999 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144497+0000", 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014351819999999999 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144516+0000", 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9267e-05 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144575+0000", 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9032e-05 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.144678+0000", 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102526 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:21741 3.a e114/107 3:549a6936:::rbd_data.11525992f601.0000000000000002:head v 114'526, mlcod=112'524)", 2024-09-17T23:31:21.440 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.1620899100000002, 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031310919999999998, 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173029+0000", 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8899999999999998e-07 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173037+0000", 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3350000000000001e-06 2024-09-17T23:31:21.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173048+0000", 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0995000000000001e-05 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173054+0000", 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5239999999999997e-06 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173133+0000", 2024-09-17T23:31:21.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9077000000000002e-05 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173155+0000", 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1630000000000001e-05 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176110+0000", 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029556130000000002 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176159+0000", 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8928999999999999e-05 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21759 3.17 3:ee28c85f:::rbd_data.11525992f601.00000000000000ad:head [delete] snapc 2c=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.1557771090000002, 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025726450000000001, 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:21.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21759 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179342+0000", 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.164e-06 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179347+0000", 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0050000000000004e-06 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179364+0000", 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7059e-05 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179373+0000", 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6580000000000008e-06 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179456+0000", 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2899999999999996e-05 2024-09-17T23:31:21.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179541+0000", 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4640000000000003e-05 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179660+0000", 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000119169 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181588+0000", 2024-09-17T23:31:21.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0019279729999999999 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181773+0000", 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001852 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181843+0000", 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0487000000000005e-05 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:21.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181914+0000", 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0389999999999995e-05 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:21.451 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:21.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:21.453 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:21.451+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:21.553 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:21.551+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:21.560 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:21.560 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:21.560 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:21.561 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:21.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:21.654 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:21.653+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:21.676 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:21.676 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:21.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:21.755 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:21.754+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.988994452, 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:21.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:21.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:21.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.979422352999997, 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:21.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:21.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.975891847, 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:21.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.974397140999997, 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:21.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:21.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.974214222000001, 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:21.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.974100720999999, 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:21.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.974076613999998, 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:21.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:21.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.974002282999997, 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:21.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:21.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:21.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.973978733999999, 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.970807532999999, 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:21.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.966399197000001, 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:21.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.939127089999999, 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:21.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:21.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:21.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:21.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.929944937999998, 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:21.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:21.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:21.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.929913444999997, 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:21.905 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:21.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.929887753000003, 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:21.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.929743201000001, 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:21.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:21.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.928175084000003, 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:21.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:21.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.927977736999999, 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:21.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.927951422, 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:21.932 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:21.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:21.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.927923079000003, 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:21.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:21.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:21.942 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:21.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:31:21.945 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:21.942+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:22.044 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:22.043+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:22.091 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:22.092 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:22.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:22.145 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:22.143+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:22.246 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:22.244+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:22.274 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:22.274 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:22.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:22.369 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:22.367+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:22.444 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.97002173300001, 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:22.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.112185920000002, 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:22.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.111493627000002, 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:22.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.111482046000006, 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.110954798999998, 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:22.467 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:22.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.110920816000004, 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:22.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.110610223999998, 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:22.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.110584090000003, 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:22.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.110086183000007, 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.109485093000004, 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:31:22.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:31:22.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.108936458000002, 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:22.494 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:22.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:22.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.108728859999999, 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:22.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:22.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.099112958999996, 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.504 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:22.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.099102724999995, 2024-09-17T23:31:22.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:22.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:22.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.099092858000006, 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:22.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.099082710000005, 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:22.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:22.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.098612841999994, 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:22.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:22.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.098590725999998, 2024-09-17T23:31:22.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:22.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:22.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12337 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.098303031, 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050465015000000002, 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:22.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12337 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343158+0000", 2024-09-17T23:31:22.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343315+0000", 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343319+0000", 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0130000000000004e-06 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343328+0000", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6179999999999995e-06 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343334+0000", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6470000000000001e-06 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393486+0000", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050151500000000002 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393546+0000", 2024-09-17T23:31:22.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9678e-05 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393623+0000", 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7743000000000004e-05 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:22.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.912916860999999, 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:22.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:22.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:22.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:22.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:22.543 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:22.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:22.545 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:22.543+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:22.645 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:22.643+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:22.697 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:22.697 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:22.698 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:22.698 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:22.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:22.752 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:22.750+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:22.830 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:22.831 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:22.831 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:22.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:22.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:22.882 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:22.880+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:22.949 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:22.947+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:23.012 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.973821272, 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:23.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:23.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:23.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.973648332, 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:23.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:23.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:23.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:23.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:23.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.973608462, 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:23.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:23.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:23.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:23.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.679104235, 2024-09-17T23:31:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:23.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:23.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:23.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:23.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.431402685999998, 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:23.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.421470420000006, 2024-09-17T23:31:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:23.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:23.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:23.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.408243370999998, 2024-09-17T23:31:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:23.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:23.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:23.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:23.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867714215000007, 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:23.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:23.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:23.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867692825000006, 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:23.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:23.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:23.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867675696000006, 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:23.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:23.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867658414000005, 2024-09-17T23:31:23.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:23.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:23.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:23.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:23.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867641384999999, 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:23.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:23.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867608511, 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:23.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:23.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.867020804999996, 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:23.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:23.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.845228664000004, 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:23.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:23.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.844691475999994, 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:23.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.844205360000004, 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:23.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:23.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.772062758000004, 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:23.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:23.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:23.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.772040946999994, 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:23.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:23.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.762598077000007, 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:23.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:23.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:23.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:23.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:23.160 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:23.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:23.162 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:23.162 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:23.162 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:23.163 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:23.163 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:23.163 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:23.164 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:23.164 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:23.164 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:23.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:23.165 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:23.159+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:23.261 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:23.260+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:23.272 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:23.273 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:23.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:23.362 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:23.361+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:23.441 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:23.485 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:23.483+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:23.553 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:23.553 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:23.563 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:23.604 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:23.602+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:23.665 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:23.664+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:23.720 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:23.720 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:23.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:23.766 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:23.765+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:23.835 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:23.835 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:23.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:23.867 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:23.865+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:23.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:31:23.950 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:23.950 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:23.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:23.990 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:23.988+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:24.066 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.067 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.069 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:24.067+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:24.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:24.170 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:24.168+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:24.227 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.228 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:24.271 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:24.269+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:24.372 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:24.370+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:24.384 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.384 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:24.473 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:24.472+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:24.502 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.502 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:24.574 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:24.573+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:24.657 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.657 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:24.711 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:24.709+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:24.774 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.774 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.776 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:24.774+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:24.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:24.878 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:24.876+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:24.930 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:24.931 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:24.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:24.983 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:24.981+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:25.046 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:25.046 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:25.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:25.080 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:25.078+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:25.105 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5, 1] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:31:25.106 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:25.106 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T23:31:25.106 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.184343 2024-09-17T23:31:25.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.18434304164786153 2024-09-17T23:31:25.107 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:31:25.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:25.180 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:25.178+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:25.222 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:25.222 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:25.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:25.281 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:25.280+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:25.305 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:25.305 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:25.323 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:25.323 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:25.339 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:25.339 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:25.342 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:25.342 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:25.343 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:25.343 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:25.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:25.374 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:25.374 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:25.382 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:25.381+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:25.391 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:25.392 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:25.460 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:25.461 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:25.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:25.484 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:25.482+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:25.578 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.539565307, 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:25.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:25.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:25.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.539392367, 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:25.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:25.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:25.591 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.539352497, 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:25.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:25.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.24484827000001, 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:25.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.997146721000007, 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:25.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:25.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.987214455, 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:25.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:25.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.973987406000006, 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:25.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:25.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.433458250000001, 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:25.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:25.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.43343686, 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.433419731000001, 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:25.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:25.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:25.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:25.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.433402448999999, 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:25.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.433385419999993, 2024-09-17T23:31:25.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.433352545999995, 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:25.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.432764840000004, 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.410972698999998, 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:25.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:25.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.410435511000003, 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:25.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.409949394999998, 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:25.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.337806792999999, 2024-09-17T23:31:25.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:25.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:25.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.337784982000002, 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.328342112000001, 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:25.687 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:25.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:25.689 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:25.686+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:25.788 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:25.787+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:25.809 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:25.809 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:25.809 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:25.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:25.856 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 0.184343 (8120812) 2024-09-17T23:31:25.915 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:25.913+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:25.933 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:25.933 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:25.933 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:25.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:25.990 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:25.989+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:26.050 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.172722088, 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.051 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:26.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.157096958, 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:26.057 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:26.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.15705930199999, 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:26.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.068 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.157041871, 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:26.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:26.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.15700694, 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:26.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:26.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.15697552, 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:26.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:26.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.156958894, 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.1569372, 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:26.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:26.096 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.15692548200001, 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:26.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.197786962999999, 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:26.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:26.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:26.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:26.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.193831798999994, 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.111 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:26.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:26.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.193791141000005, 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.119 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:26.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.124 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.193531715999995, 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:26.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:26.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.193495494000004, 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.130 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:26.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:26.135 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.193463199000007, 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.136 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:26.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.193308388000005, 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:26.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 60.193022067999998, 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:26.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:26.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 60.192953406999997, 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:26.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:26.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.157 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 60.192911432999999, 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:26.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.161 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.1931274099999998, 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:26.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:26.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:26.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.169 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.169 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.169 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:26.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:26.171 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:26.168+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:26.271 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:26.269+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:26.321 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:26.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:26.322 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:26.322 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:26.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:26.371 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:26.369+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:26.434 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:26.434 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:26.434 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:26.434 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:26.434 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:26.443 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:26.484 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:26.482+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:26.573 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:26.572+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:26.591 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:26.591 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:26.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.09443994999999, 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:26.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:26.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.09442007, 2024-09-17T23:31:26.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:26.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.094398532, 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.601 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:26.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:26.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.09251593400001, 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:26.606 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.09250581800001, 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:26.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.092493588, 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:26.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.092483019, 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.620 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:26.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:26.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.624 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.09247313100001, 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:26.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.092462849, 2024-09-17T23:31:26.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:26.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:26.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.092446962, 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:26.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 88.553521645000004, 2024-09-17T23:31:26.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:26.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:26.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.644 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 88.551452259000001, 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:26.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.650 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.219584890999997, 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:26.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.734620212999999, 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.657 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:26.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:26.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.734587843, 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.662 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:26.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.734295466000006, 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:26.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:26.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.728471327999998, 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.673 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:26.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:26.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:26.678 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.678 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.728429825000006, 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:26.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.685691558000002, 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:26.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:26.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.685589217, 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:26.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:26.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:26.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:26.698 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:26.698 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:26.699 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:26.700 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:26.697+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:26.800 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:26.799+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:26.806 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:26.806 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:26.807 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:26.807 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:26.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:26.902 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:26.900+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:26.924 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:26.924 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:26.924 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:26.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:27.018 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:27.016+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:27.084 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "MOSDPGPush(3.a 108/107 [PushOp(3:536e2895:::rbd_data.1152c10d4b9c.000000000000000c:2a, version: 103'497, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:536e2895:::rbd_data.1152c10d4b9c.000000000000000c:2a@103'497, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 2a=[]:{2a: [2a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:27.085 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.822931861000001, 2024-09-17T23:31:27.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002379188, 2024-09-17T23:31:27.086 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:27.086 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:27.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258222+0000", 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.258223+0000", 2024-09-17T23:31:27.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.82e-07 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259145+0000", 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00092170299999999995 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259163+0000", 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8284e-05 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259176+0000", 2024-09-17T23:31:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2747e-05 2024-09-17T23:31:27.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259259+0000", 2024-09-17T23:31:27.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2981999999999999e-05 2024-09-17T23:31:27.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.259271+0000", 2024-09-17T23:31:27.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2711000000000001e-05 2024-09-17T23:31:27.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:12.260601+0000", 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013301789999999999 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:20738 3.1c e109/108 3:3831950c:::rbd_directory:head v 109'350, mlcod=0'0)", 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.367249687999999, 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026884439999999999, 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713904+0000", 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713905+0000", 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7599999999999997e-07 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713910+0000", 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3380000000000004e-06 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713922+0000", 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1185999999999999e-05 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713927+0000", 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.524e-06 2024-09-17T23:31:27.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713963+0000", 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6369999999999999e-05 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.713981+0000", 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7986e-05 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.716542+0000", 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025610540000000001 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:13.716593+0000", 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.041e-05 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1011 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 5636096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:27.098 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.968052459000001, 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025798650000000002, 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:27.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1011 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113102+0000", 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113103+0000", 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.708e-06 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113110+0000", 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0160000000000003e-06 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113127+0000", 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7019000000000001e-05 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113135+0000", 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6559999999999992e-06 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113185+0000", 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0204999999999998e-05 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.113189+0000", 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9249999999999997e-06 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114714+0000", 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0015248060000000001 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114759+0000", 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5420000000000002e-05 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.114842+0000", 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2470000000000002e-05 2024-09-17T23:31:27.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115485+0000", 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00064293800000000002 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115630+0000", 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000145301 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115659+0000", 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9251000000000002e-05 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.115682+0000", 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.215e-05 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1209 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 15597568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.106 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.876426006999999, 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044407630000000003, 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1209 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204729+0000", 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.077e-06 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204736+0000", 2024-09-17T23:31:27.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7889999999999997e-06 2024-09-17T23:31:27.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204747+0000", 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0793e-05 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204754+0000", 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3379999999999999e-06 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204820+0000", 2024-09-17T23:31:27.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6235000000000005e-05 2024-09-17T23:31:27.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204911+0000", 2024-09-17T23:31:27.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1024000000000001e-05 2024-09-17T23:31:27.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:27.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.205182+0000", 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000270134 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.208700+0000", 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035188810000000002 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209111+0000", 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00041066099999999999 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209144+0000", 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2653000000000001e-05 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209169+0000", 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5177999999999998e-05 2024-09-17T23:31:27.115 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1629 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.701702205, 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038611639999999998, 2024-09-17T23:31:27.116 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1629 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379453+0000", 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1429999999999999e-06 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379457+0000", 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0969999999999999e-06 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379469+0000", 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1868999999999999e-05 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379476+0000", 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2400000000000001e-06 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379533+0000", 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6666999999999998e-05 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379584+0000", 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1057000000000003e-05 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379647+0000", 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2778000000000001e-05 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380452+0000", 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080491199999999997 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383214+0000", 2024-09-17T23:31:27.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002761922 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383272+0000", 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8184999999999999e-05 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383313+0000", 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1294000000000001e-05 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1624 3.11 e112/107 3:8fa7046a:::rbd_data.1152d7c8a97e.000000000000000a:head v 112'327, mlcod=112'325)", 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.701131428, 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031108070000000002, 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380023+0000", 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7000000000000004e-07 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380030+0000", 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.455e-06 2024-09-17T23:31:27.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380041+0000", 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0786e-05 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380048+0000", 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380105+0000", 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6938000000000003e-05 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380124+0000", 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9332000000000001e-05 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383100+0000", 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002975924 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383133+0000", 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3688e-05 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.701086794, 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:27.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1643 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14811136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.693452655, 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042815140000000002, 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1643 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387703+0000", 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.308e-06 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387708+0000", 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.865e-06 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387723+0000", 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.556e-05 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387732+0000", 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5099999999999998e-06 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387795+0000", 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2929999999999995e-05 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387897+0000", 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102815 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388033+0000", 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013546899999999999 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388921+0000", 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088850099999999998 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391906+0000", 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002984202 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391944+0000", 2024-09-17T23:31:27.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8565000000000001e-05 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391983+0000", 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8788999999999997e-05 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.143 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:2237 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1554432~18432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.442045834, 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021808159999999999, 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.144 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2237 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639108+0000", 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639110+0000", 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6640000000000001e-06 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639124+0000", 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4316e-05 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639135+0000", 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1134e-05 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639141+0000", 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8379999999999998e-06 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639200+0000", 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9005999999999997e-05 2024-09-17T23:31:27.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639286+0000", 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6138999999999996e-05 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.639431+0000", 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014437799999999999 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641204+0000", 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017734669999999999 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641230+0000", 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5428000000000001e-05 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641263+0000", 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3337000000000003e-05 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.641289+0000", 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6109e-05 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21422 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.100404338000001, 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044109429999999996, 2024-09-17T23:31:27.150 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21422 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980751+0000", 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980755+0000", 2024-09-17T23:31:27.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:31:27.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980768+0000", 2024-09-17T23:31:27.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2862e-05 2024-09-17T23:31:27.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980776+0000", 2024-09-17T23:31:27.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8359999999999992e-06 2024-09-17T23:31:27.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980833+0000", 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6749000000000002e-05 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.155 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980879+0000", 2024-09-17T23:31:27.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6050999999999997e-05 2024-09-17T23:31:27.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:27.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980977+0000", 2024-09-17T23:31:27.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8488999999999998e-05 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982120+0000", 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001143257 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985096+0000", 2024-09-17T23:31:27.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00297569 2024-09-17T23:31:27.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985133+0000", 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7134000000000001e-05 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985161+0000", 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7514000000000001e-05 2024-09-17T23:31:27.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:27.160 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.098835742, 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.161 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:27.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:27.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.998203105, 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:27.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:27.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:21741 3.a e114/107 3:549a6936:::rbd_data.11525992f601.0000000000000002:head v 114'526, mlcod=112'524)", 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.9081258040000009, 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031310919999999998, 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173029+0000", 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8899999999999998e-07 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173037+0000", 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3350000000000001e-06 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173048+0000", 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0995000000000001e-05 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173054+0000", 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5239999999999997e-06 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173133+0000", 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9077000000000002e-05 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173155+0000", 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1630000000000001e-05 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176110+0000", 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029556130000000002 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176159+0000", 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8928999999999999e-05 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21759 3.17 3:ee28c85f:::rbd_data.11525992f601.00000000000000ad:head [delete] snapc 2c=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:27.180 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.9018130030000009, 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025726450000000001, 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21759 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179341+0000", 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179342+0000", 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.164e-06 2024-09-17T23:31:27.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179347+0000", 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0050000000000004e-06 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179364+0000", 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7059e-05 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179373+0000", 2024-09-17T23:31:27.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6580000000000008e-06 2024-09-17T23:31:27.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179456+0000", 2024-09-17T23:31:27.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2899999999999996e-05 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179541+0000", 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4640000000000003e-05 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:27.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.179660+0000", 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000119169 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181588+0000", 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0019279729999999999 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181773+0000", 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001852 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181843+0000", 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0487000000000005e-05 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.181914+0000", 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0389999999999995e-05 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.187 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22085 3.0 3:04b93b6a:::rbd_data.1152c10d4b9c.0000000000000002:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.230836+0000", 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8503185100000001, 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0030537989999999998, 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.188 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22085 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230836+0000", 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230836+0000", 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230838+0000", 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7e-06 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230850+0000", 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1293e-05 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230883+0000", 2024-09-17T23:31:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3340000000000003e-05 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230898+0000", 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5407e-05 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.231023+0000", 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012492100000000001 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.231208+0000", 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00018450500000000001 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.231386+0000", 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000177973 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233135+0000", 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001749219 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233677+0000", 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00054229199999999999 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233718+0000", 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1066000000000003e-05 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233889+0000", 2024-09-17T23:31:27.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017108299999999999 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22086 3.0 3:04a20ea7:::rbd_data.1152c10d4b9c.000000000000000b:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8502165350000004, 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0043112539999999996, 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.194 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22086 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:27.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230939+0000", 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3370000000000001e-06 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230945+0000", 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.37e-06 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230966+0000", 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0492999999999998e-05 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230991+0000", 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5463e-05 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232196+0000", 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012043069999999999 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232366+0000", 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017094000000000001 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232576+0000", 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020904100000000001 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233937+0000", 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013611140000000001 2024-09-17T23:31:27.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235062+0000", 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001125175 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235117+0000", 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4976999999999999e-05 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235249+0000", 2024-09-17T23:31:27.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000132037 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22095 3.0 3:03401d0b:::rbd_data.1152c10d4b9c.000000000000001d:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8472016350000002, 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003963388, 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.200 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22095 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233954+0000", 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.181e-06 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233959+0000", 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4569999999999996e-06 2024-09-17T23:31:27.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233971+0000", 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1924e-05 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233976+0000", 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4310000000000002e-06 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234018+0000", 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1050000000000002e-05 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234197+0000", 2024-09-17T23:31:27.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017971 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234374+0000", 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017669300000000001 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235543+0000", 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011695060000000001 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237714+0000", 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021710039999999998 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237765+0000", 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0853999999999998e-05 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237916+0000", 2024-09-17T23:31:27.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015057800000000001 2024-09-17T23:31:27.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.207 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.207 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.207 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8456754589999997, 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.208 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.209 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.209 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:31:27.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.209 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:27.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:27.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:31:27.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:31:27.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:31:27.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:31:27.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:31:27.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:31:27.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:31:27.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:31:27.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:31:27.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:31:27.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:31:27.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:31:27.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:27.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:31:27.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:31:27.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:31:27.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:31:27.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:31:27.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:31:27.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:31:27.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:31:27.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:31:27.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:31:27.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.221 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.222 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:27.222 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:27.222 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8455382199999999, 2024-09-17T23:31:27.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:31:27.222 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:31:27.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.224 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:27.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:27.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:31:27.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:31:27.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:31:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:31:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.228 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:31:27.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:31:27.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.228 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:31:27.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:31:27.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:31:27.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:31:27.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:31:27.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:31:27.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:27.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:31:27.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:31:27.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:31:27.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:31:27.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:31:27.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:31:27.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:31:27.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:31:27.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:31:27.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:31:27.235 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.235 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.236 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.236 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:27.236 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:27.237 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.8403810600000003, 2024-09-17T23:31:27.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:31:27.237 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:27.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:27.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:27.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:27.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:27.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:27.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:31:27.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.238 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:27.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:27.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:27.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:27.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:27.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:27.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:27.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:31:27.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:31:27.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:27.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:31:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:31:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:31:27.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:31:27.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:27.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:31:27.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:31:27.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:27.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:31:27.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:31:27.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:27.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:31:27.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:31:27.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:31:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:31:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:31:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:31:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:27.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:31:27.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:31:27.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:27.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:31:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:31:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:31:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:31:27.250 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.250 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.250 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.250 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:27.250 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:27.250 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:27.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:31:27.253 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:27.250+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:27.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:27.353 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:27.351+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:27.375 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:27.376 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:27.376 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:27.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:27.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:27.454 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:27.452+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:27.495 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:27.495 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:27.495 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:27.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:27.555 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:27.553+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:27.556 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:27.556 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:27.574 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:27.575 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:27.591 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:27.608 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:27.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:27.625 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:27.642 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:27.643 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:27.656 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:27.654+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:27.733 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:27.733 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:27.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:27.756 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:27.755+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:27.849 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:27.849 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:31:27.857 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:27.855+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:27.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:27.958 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:27.956+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:27.974 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:27.975 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:27.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:28.059 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:28.057+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:28.092 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.092 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:28.160 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:28.159+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:28.248 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.249 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:28.301 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:28.299+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:28.362 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:28.360+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:28.365 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.367 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:28.463 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:28.461+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:28.492 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.492 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:28.586 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:28.584+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:28.650 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.650 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:28.700 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:28.698+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:28.766 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:28.764+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:28.806 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.806 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:28.866 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:28.865+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:28.929 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:28.929 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:28.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:28.983 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:28.981+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:29.049 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:29.049 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:29.058 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:29.069 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:29.067+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:29.170 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:29.168+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:29.205 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:29.206 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:29.214 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:29.270 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:29.269+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:29.362 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:29.362 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:29.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:29.372 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:29.370+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:29.472 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:29.471+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:29.477 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:29.477 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:29.487 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:29.574 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:29.572+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:29.594 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:29.595 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:29.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:29.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:29.699 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:29.697+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:29.722 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:29.722 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:29.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:29.775 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:29.773+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:29.782 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:29.782 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:29.800 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:29.800 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:29.816 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:29.817 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:29.834 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:29.834 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:29.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:29.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:29.868 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:29.868 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:29.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:31:29.877 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:29.875+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:29.886 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:29.886 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:29.886 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:29.886 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:29.886 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:29.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:29.982 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:29.981+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:30.060 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.215467720999996, 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.205895622, 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.202365115999996, 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.200870409999993, 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.200687490999996, 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.200573989999995, 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:30.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.200549882999994, 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:30.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:30.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.200475552, 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:30.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:30.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.200452002999995, 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.197280801999995, 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:30.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:30.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.192872465999997, 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.165600358999995, 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:30.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:30.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.156418207000002, 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:30.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:30.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.131 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.156386714000007, 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:30.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.156361021999999, 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:30.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:30.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.156216470000004, 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:30.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:30.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.154648352999999, 2024-09-17T23:31:30.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:30.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:30.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.154451006000002, 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:30.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.154424691000003, 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:30.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.154396348000006, 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:30.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:30.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.169 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:30.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:30.171 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:30.168+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:30.271 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:30.269+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:30.318 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:30.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:30.319 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:30.319 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:30.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:30.412 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:30.410+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:30.472 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:30.470+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:30.475 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:30.475 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:30.475 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:30.475 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:30.475 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:30.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:30.573 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:30.571+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:30.633 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:30.633 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 151.15906388900001, 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:30.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:30.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:30.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:30.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.301228076000001, 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:30.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:30.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.300535783000001, 2024-09-17T23:31:30.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:30.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.300524202000005, 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:30.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:30.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.299996954999997, 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:30.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.299962972000003, 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:30.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:30.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.299652379999998, 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:30.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.299626246000003, 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.299128339000006, 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.298527249000003, 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:31:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:31:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.297978614000002, 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.297771015999999, 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.288155114999995, 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.288144880999994, 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.288135014000005, 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.288124866000004, 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.287654997999994, 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.287632881999997, 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.101959016999999, 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.72836239, 2024-09-17T23:31:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:31:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:30.730 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:30.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:30.732 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:30.729+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:30.832 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:30.831+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:30.874 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5, 1] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:31:30.874 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:30.874 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T23:31:30.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-17T23:31:30.880 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:30.880 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:30.881 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:30.881 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:30.881 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:31:30.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:30.942 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:30.940+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:31.034 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:31.033+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:31.054 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:31.054 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:31.054 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:31.054 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:31.054 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:31.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:31.147 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:31.146+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:31.180 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:31:31.178+0000 7f44fd5a2640 -1 mgr.server reply reply (3) No such process no change 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:moved 8 / 82 (9.7561%) 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.23832 -> 2.8 (expected baseline 3.62215) 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 8 -> 14 pgs (0.487805 -> 0.853659 * mean) 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 21 pgs (1.46341 -> 1.28049 * mean) 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:oload 1000 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0007 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0067 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.6360 -> 1.0000 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.8582 -> 0.8978 2024-09-17T23:31:31.181 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 weight 0.8419 -> 0.8597 2024-09-17T23:31:31.182 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-17T23:31:31.193 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T23:31:31.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-17T23:31:31.213 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.174736713, 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.174563773, 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.17452390299999, 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 110.880019676, 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.632318127, 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.622385860999998, 2024-09-17T23:31:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.609158812000004, 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.068629655999999, 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.068608265999998, 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.068591136999999, 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.068573854999997, 2024-09-17T23:31:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.068556826000005, 2024-09-17T23:31:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.068523952000007, 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.067936246000002, 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.046144104999996, 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.045606917000001, 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.045120800999996, 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.972978198999996, 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.972956388, 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.963513517999999, 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:31.312 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:31.313 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:31.314 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:31.311+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:31.414 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:31.412+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:31.426 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:31.426 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:31.426 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:31.426 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:31.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:31.515 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:31.513+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:31.542 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:31.542 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:31.543 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:31.543 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:31.543 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:31.551 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:31.599 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:31:31.597+0000 7f44fd5a2640 -1 mgr.server reply reply (4) Interrupted system call no change 2024-09-17T23:31:31.600 INFO:teuthology.orchestra.run.smithi031.stdout:moved 15 / 82 (18.2927%) 2024-09-17T23:31:31.600 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:31:31.600 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.23832 -> 1.62481 (expected baseline 3.62215) 2024-09-17T23:31:31.600 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 8 -> 18 pgs (0.487805 -> 1.09756 * mean) 2024-09-17T23:31:31.600 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 14 pgs (1.46341 -> 0.853659 * mean) 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0007 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0007 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.6360 -> 1.0000 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 weight 1.0000 -> 0.7954 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.8582 -> 0.8978 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 1.0000 -> 0.7710 2024-09-17T23:31:31.601 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-17T23:31:31.613 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T23:31:31.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-17T23:31:31.616 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:31.614+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:31.700 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.822915587, 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:31.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:31.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:31.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.80729045699999, 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:31.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.807252801, 2024-09-17T23:31:31.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:31.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:31.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.80723537, 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.719 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:31.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.807200439, 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:31.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.807169019, 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:31.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:31.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.734 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.807152393, 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:31.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.807130699, 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:31.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:31.744 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 124.807118981, 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:31.745 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:31.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:31.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.847980461999995, 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.750 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:31.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:31.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:31.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:31.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:31.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.844025298000005, 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.757 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:31.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.843984640000002, 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:31.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:31.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.843725215000006, 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:31.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:31.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.772 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.843688993000001, 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:31.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:31.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:31.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.843656698000004, 2024-09-17T23:31:31.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:31.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 70.843501887000002, 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.784 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.843215567000001, 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.789 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:31.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.793 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.843146906000001, 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.794 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:31.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:31.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:31.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.798 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 65.843104932000003, 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.802 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.803 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 10.843320908999999, 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:31.804 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:31.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:31.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:31.807 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:31.809 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.810 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.810 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:31.810 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:31.810 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:31.811 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:31.812 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:31.809+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:31.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:31.912 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:31.910+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:31.955 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:31:31.953+0000 7f44fd5a2640 -1 mgr.server reply reply (3) No such process no change 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:moved 8 / 82 (9.7561%) 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.23832 -> 2.8 (expected baseline 3.62215) 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 8 -> 14 pgs (0.487805 -> 0.853659 * mean) 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 21 pgs (1.46341 -> 1.28049 * mean) 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:oload 1000 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:31:31.956 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0007 2024-09-17T23:31:31.957 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0067 2024-09-17T23:31:31.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.6360 -> 1.0000 2024-09-17T23:31:31.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.8582 -> 0.8978 2024-09-17T23:31:31.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 weight 0.8419 -> 0.8597 2024-09-17T23:31:31.957 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-17T23:31:31.969 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T23:31:31.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T23:31:31.972 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:31.972 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:31.973 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:31.973 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:31.984 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:32.012 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:32.011+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:32.022 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:32.022 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:32.040 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:32.057 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:32.074 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:32.074 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:32.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:32.093 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:32.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:32.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:32.113 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:32.111+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:32.132 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:32.132 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:32.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:32.214 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:32.213+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:32.289 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:32.290 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:32.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:32.343 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:32.341+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:32.406 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:32.406 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:32.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:32.416 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:32.415+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:32.517 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:32.515+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:32.562 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:32.562 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:32.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:32.618 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:32.616+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:32.718 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:32.719 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:32.719 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:32.717+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:32.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:32.820 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:32.818+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:32.843 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:32.843 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:32.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:32.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:31:32.904 INFO:teuthology.orchestra.run.smithi031.stdout:moved 15 / 82 (18.2927%) 2024-09-17T23:31:32.904 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:31:32.904 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 5.23832 -> 1.62481 (expected baseline 3.62215) 2024-09-17T23:31:32.904 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.5 with 8 -> 18 pgs (0.487805 -> 1.09756 * mean) 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.0 with 24 -> 14 pgs (1.46341 -> 0.853659 * mean) 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0007 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0007 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 weight 0.6360 -> 1.0000 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 weight 1.0000 -> 0.7954 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.8582 -> 0.8978 2024-09-17T23:31:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 1.0000 -> 0.7710 2024-09-17T23:31:32.921 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:32.919+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:32.927 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T23:31:32.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-17T23:31:33.010 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.010 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:33.021 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:33.019+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:33.123 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:33.121+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:33.171 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.172 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:33.224 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:33.222+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:33.266 INFO:tasks.ceph.mgr.x.smithi037.stderr:2024-09-17T23:31:33.264+0000 7f44fd5a2640 -1 mgr.server reply reply (4) Interrupted system call no change 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout:moved 13 / 82 (15.8537%) 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout:avg 16.4 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 1.62481 -> 4.88262 (expected baseline 3.62215) 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.0 with 14 -> 9 pgs (0.853659 -> 0.548781 * mean) 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.4 with 18 -> 23 pgs (1.09756 -> 1.40244 * mean) 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:33.267 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0007 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0007 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 weight 0.7954 -> 0.6326 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 weight 0.8978 -> 0.9392 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 weight 0.7710 -> 0.5944 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 weight 0.8419 -> 0.8597 2024-09-17T23:31:33.268 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-17T23:31:33.294 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.295 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:33.345 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:33.343+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:33.426 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:33.425+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:33.451 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.451 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:33.527 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:33.526+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:33.567 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.567 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:33.659 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:33.658+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:33.724 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.724 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.729 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:33.727+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:33.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:33.830 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:33.829+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:33.880 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:33.880 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:33.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:33.915 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:31:33.932 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:33.930+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:34.022 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:34.022 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:34.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:34.032 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:34.030+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:34.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:34.133 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:34.131+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:34.150 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:34.150 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:34.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:34.234 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:34.232+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:34.245 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:34.245 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:34.262 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:34.262 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:34.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:34.295 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:34.295 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:34.309 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:34.312 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:34.312 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:34.312 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:34.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:34.328 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:34.328 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:34.334 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:34.333+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:34.427 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.549918181, 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.53429305100001, 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.534255395, 2024-09-17T23:31:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.534237964, 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.534203033, 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.534171613, 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.53415498699999, 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.534133293, 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 127.534121575, 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.574983055999994, 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.571027892000004, 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.570987234, 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.570727809000005, 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.570691586999999, 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.570659292000002, 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.570504481, 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.570218161, 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.570149499999999, 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 68.570107526000001, 2024-09-17T23:31:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.570323503000001, 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.548 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:34.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:34.550 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:34.548+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:34.651 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:34.649+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:34.657 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:34.658 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:34.666 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:34.751 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:34.750+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:34.773 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:34.773 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:34.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:34.852 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:34.851+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.393036044, 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.393016164, 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.392994626, 2024-09-17T23:31:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2119 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1216512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.39111202799999, 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049234829000000001, 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2119 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495939+0000", 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.22e-07 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495941+0000", 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495944+0000", 2024-09-17T23:31:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1769999999999998e-06 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495946+0000", 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.544997+0000", 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049050752000000003 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545082+0000", 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4547000000000002e-05 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545174+0000", 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2538000000000001e-05 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.391101912, 2024-09-17T23:31:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.391089682, 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.391079113, 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.391069225, 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.391058943, 2024-09-17T23:31:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 117.391043056, 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.852117738999993, 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.850048353000005, 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 95.518180985000001, 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.033216307000004, 2024-09-17T23:31:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.033183937000004, 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.032891559999996, 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.027067422000002, 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.027025918999996, 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.984287652000006, 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 73.984185311000005, 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:34.989 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:34.990 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:34.991 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:34.988+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:35.092 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:35.089+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:35.122 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:35.122 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:35.133 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:35.191 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:35.190+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:35.292 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:35.290+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:35.293 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:35.293 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:35.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:35.393 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:35.391+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1209 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 15597568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 19.209218667999998, 2024-09-17T23:31:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044407630000000003, 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1209 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204729+0000", 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.077e-06 2024-09-17T23:31:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204736+0000", 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7889999999999997e-06 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204747+0000", 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0793e-05 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204754+0000", 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3379999999999999e-06 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204820+0000", 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6235000000000005e-05 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204911+0000", 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1024000000000001e-05 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.205182+0000", 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000270134 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.208700+0000", 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035188810000000002 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209111+0000", 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00041066099999999999 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209144+0000", 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2653000000000001e-05 2024-09-17T23:31:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209169+0000", 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5177999999999998e-05 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1629 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 19.034494865999999, 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038611639999999998, 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1629 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379453+0000", 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1429999999999999e-06 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379457+0000", 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0969999999999999e-06 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379469+0000", 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1868999999999999e-05 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379476+0000", 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2400000000000001e-06 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379533+0000", 2024-09-17T23:31:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6666999999999998e-05 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379584+0000", 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1057000000000003e-05 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379647+0000", 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2778000000000001e-05 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380452+0000", 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080491199999999997 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383214+0000", 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002761922 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383272+0000", 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8184999999999999e-05 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383313+0000", 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1294000000000001e-05 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 19.033879455000001, 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1643 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14811136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 19.026245316000001, 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042815140000000002, 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1643 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387703+0000", 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.308e-06 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387708+0000", 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.865e-06 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387723+0000", 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.556e-05 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387732+0000", 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5099999999999998e-06 2024-09-17T23:31:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387795+0000", 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2929999999999995e-05 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387897+0000", 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102815 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388033+0000", 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013546899999999999 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388921+0000", 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088850099999999998 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391906+0000", 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002984202 2024-09-17T23:31:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391944+0000", 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8565000000000001e-05 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391983+0000", 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8788999999999997e-05 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21422 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.433196999, 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044109429999999996, 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21422 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980751+0000", 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980755+0000", 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980768+0000", 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2862e-05 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980776+0000", 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8359999999999992e-06 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980833+0000", 2024-09-17T23:31:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6749000000000002e-05 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980879+0000", 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6050999999999997e-05 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980977+0000", 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8488999999999998e-05 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982120+0000", 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001143257 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985096+0000", 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00297569 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985133+0000", 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7134000000000001e-05 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985161+0000", 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7514000000000001e-05 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.431628403000001, 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.330995766000001, 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:21741 3.a e114/107 3:549a6936:::rbd_data.11525992f601.0000000000000002:head v 114'526, mlcod=112'524)", 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.240918465, 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0031310919999999998, 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173028+0000", 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173029+0000", 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8899999999999998e-07 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173037+0000", 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3350000000000001e-06 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173048+0000", 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0995000000000001e-05 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173054+0000", 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5239999999999997e-06 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173133+0000", 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9077000000000002e-05 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.173155+0000", 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.1630000000000001e-05 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176110+0000", 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0029556130000000002 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.176159+0000", 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8928999999999999e-05 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22086 3.0 3:04a20ea7:::rbd_data.1152c10d4b9c.000000000000000b:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.183009196, 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0043112539999999996, 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22086 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230939+0000", 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3370000000000001e-06 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230945+0000", 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.37e-06 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230966+0000", 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0492999999999998e-05 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230991+0000", 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5463e-05 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232196+0000", 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012043069999999999 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232366+0000", 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017094000000000001 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232576+0000", 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020904100000000001 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233937+0000", 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013611140000000001 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235062+0000", 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001125175 2024-09-17T23:31:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235117+0000", 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4976999999999999e-05 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235249+0000", 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000132037 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22095 3.0 3:03401d0b:::rbd_data.1152c10d4b9c.000000000000001d:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.179994296, 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003963388, 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22095 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233954+0000", 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.181e-06 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233959+0000", 2024-09-17T23:31:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4569999999999996e-06 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233971+0000", 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1924e-05 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233976+0000", 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4310000000000002e-06 2024-09-17T23:31:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234018+0000", 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1050000000000002e-05 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234197+0000", 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017971 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234374+0000", 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017669300000000001 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235543+0000", 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011695060000000001 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237714+0000", 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021710039999999998 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237765+0000", 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0853999999999998e-05 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237916+0000", 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015057800000000001 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.17846812, 2024-09-17T23:31:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:31:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:31:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:31:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.178330881000001, 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:31:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:31:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.173173721, 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:31:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:31:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22907 3.0 3:07f1f9a0:::rbd_data.11524e94aa61.0000000000000007:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T23:31:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.438119736, 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053158520000000003, 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22907 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975829+0000", 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975838+0000", 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3260000000000002e-06 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975859+0000", 2024-09-17T23:31:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0378999999999999e-05 2024-09-17T23:31:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975870+0000", 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1488000000000001e-05 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975948+0000", 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199000000000001e-05 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976110+0000", 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000161977 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976234+0000", 2024-09-17T23:31:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012316599999999999 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.977277+0000", 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010435679999999999 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981028+0000", 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037512130000000002 2024-09-17T23:31:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981092+0000", 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3468999999999998e-05 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981143+0000", 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1017999999999998e-05 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22969 3.17 3:e879c454:::rbd_header.1152324fa224:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.524950295, 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058282270000000001, 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22969 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888998+0000", 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889005+0000", 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000004e-06 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889017+0000", 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2478e-05 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889029+0000", 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.168e-05 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889311+0000", 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028192400000000002 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889435+0000", 2024-09-17T23:31:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000124046 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889886+0000", 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00045141000000000002 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.892147+0000", 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002260813 2024-09-17T23:31:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894710+0000", 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025627710000000001 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894768+0000", 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8059000000000001e-05 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894825+0000", 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6986e-05 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23066 3.1f 3:fea969d6:::rbd_data.1152324fa224.0000000000000007:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.4413142780000001, 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048693310000000002, 2024-09-17T23:31:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23066 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.569999999999999e-07 2024-09-17T23:31:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972638+0000", 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8040000000000001e-06 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972649+0000", 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0882e-05 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972656+0000", 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973702+0000", 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010456809999999999 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973830+0000", 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128071 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973960+0000", 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013065799999999999 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.975260+0000", 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012997729999999999 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977401+0000", 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021405030000000002 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977457+0000", 2024-09-17T23:31:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168999999999997e-05 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977502+0000", 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4919000000000001e-05 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23135 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'391, mlcod=0'0)", 2024-09-17T23:31:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.31073842899999998, 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053043279999999996, 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103209+0000", 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103215+0000", 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999998e-06 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103222+0000", 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4919999999999996e-06 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103231+0000", 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6049999999999998e-06 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103277+0000", 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6641999999999998e-05 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103302+0000", 2024-09-17T23:31:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4601e-05 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108442+0000", 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051404129999999999 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108513+0000", 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0192999999999997e-05 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23188 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'395, mlcod=23'1)", 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.161308+0000", 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.2526388, 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003311109, 2024-09-17T23:31:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161308+0000", 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161308+0000", 2024-09-17T23:31:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161309+0000", 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9800000000000003e-07 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161315+0000", 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7239999999999997e-06 2024-09-17T23:31:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161327+0000", 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2218000000000001e-05 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161333+0000", 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5100000000000004e-06 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161374+0000", 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1196999999999999e-05 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161391+0000", 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6620999999999999e-05 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.164576+0000", 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003185289 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.164619+0000", 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2752e-05 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23195 3.1c e131/130 3:3831950c:::rbd_directory:head v 131'414, mlcod=0'0)", 2024-09-17T23:31:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.22883272900000001, 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047496070000000003, 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185115+0000", 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1900000000000003e-07 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185119+0000", 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9230000000000001e-06 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185128+0000", 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0530000000000007e-06 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185132+0000", 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3889999999999998e-06 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185180+0000", 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8059000000000002e-05 2024-09-17T23:31:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185205+0000", 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4865999999999999e-05 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.189830+0000", 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046256379999999996 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.189864+0000", 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3160000000000001e-05 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23232 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'398, mlcod=23'1)", 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.182201539, 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048635629999999996, 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231746+0000", 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999996e-07 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231752+0000", 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7790000000000001e-06 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231760+0000", 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6279999999999994e-06 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231765+0000", 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8919999999999999e-06 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231809+0000", 2024-09-17T23:31:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3544000000000002e-05 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231837+0000", 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7815999999999999e-05 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236575+0000", 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047384030000000004 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236609+0000", 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3896999999999997e-05 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:35.554 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:35.556 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:35.554+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:35.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:35.657 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:35.655+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:35.671 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:35.671 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:35.672 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:35.672 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:35.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:35.757 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:35.756+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:35.792 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:35.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:35.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:35.885 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:35.884+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:35.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:31:35.965 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.120825628999995, 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:35.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:35.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:35.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.111253529999999, 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.107723023999995, 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:35.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.106228318000007, 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:35.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.106045398999996, 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:35.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:35.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.105931897999994, 2024-09-17T23:31:35.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:35.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:35.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.105907791000007, 2024-09-17T23:31:35.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:35.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:35.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:35.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.105833459999999, 2024-09-17T23:31:36.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:36.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.105809910999994, 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:36.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.102638709999994, 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:36.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:36.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.098230373999996, 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.070958266999995, 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:36.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:36.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:36.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:36.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.061776115000001, 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:36.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.061744622000006, 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:36.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.061718929999998, 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.061574378000003, 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:36.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:36.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.060006260999998, 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:36.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.059808914000001, 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:36.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:36.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.059782599000002, 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:36.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.059754256000005, 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:36.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:36.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:36.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.070 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:36.071 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:35.963+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:36.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:36.073 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:36.071+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:36.173 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:36.171+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:36.223 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:36.223 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:23283 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e132)", 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:35.888240+0000", 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.33389856899999998, 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.33392911600000003, 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23283 2024-09-17T23:31:36.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888240+0000", 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888240+0000", 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:36.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888242+0000", 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5120000000000001e-06 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888247+0000", 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4e-06 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888257+0000", 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0098999999999999e-05 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888263+0000", 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2089999999999996e-06 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888310+0000", 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7360000000000001e-05 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:35.888317+0000", 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6499999999999999e-06 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:36.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:36.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:36.228 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-17T23:31:36.228 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:36.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:36.275 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:36.273+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:36.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:36.341 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:36.341 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:36.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:36.375 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:36.373+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:36.475 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:36.476 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:36.474+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:36.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:36.493 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:36.508 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:36.508 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:36.510 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:36.510 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:36.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:36.527 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:36.544 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:36.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:36.561 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:36.614 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:36.612+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:36.625 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:36.625 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:36.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:36.718 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:36.717+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:36.779 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:36.777+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:36.787 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:36.787 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:36.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:31:36.880 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:36.878+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:36.954 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:36.954 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:36.963 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:36.981 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:36.980+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:37.070 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.071 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:37.082 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:37.080+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:37.185 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:37.183+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:37.186 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.186 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:37.285 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:37.283+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:37.301 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.301 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.310 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:37.387 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:37.385+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:37.457 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.458 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.466 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:37.510 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:37.508+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:37.572 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.573 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.581 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:37.589 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:37.587+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:37.690 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:37.688+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:37.797 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.798 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.798 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:37.800 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:37.795+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:37.897 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:37.896+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:37.959 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:37.959 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:37.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:37.999 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:37.997+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:38.100 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:38.098+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:38.117 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:38.118 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:38.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:38.201 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:38.199+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:38.232 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:38.233 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:38.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:38.285 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5, 1] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:31:38.285 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:38.285 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 3, 4, 5, 1] 2024-09-17T23:31:38.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-17T23:31:38.301 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:38.300+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:38.357 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:38.357 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:38.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:38.402 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:38.400+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:38.475 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:38.475 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:38.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:38.504 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:38.502+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:38.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:38.589 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:31:38.588+0000 7f21c94c5640 -1 mon.a@0(leader).osd e134 definitely_dead 0 2024-09-17T23:31:38.594 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:38.595 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:38.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:38.644 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:38.642+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:38.699 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:38.699 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:38.705 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:38.703+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:38.717 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:38.717 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:38.733 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:38.734 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:38.750 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:38.751 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:38.753 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:38.753 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:38.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:38.768 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:38.768 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:38.785 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:38.785 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:38.806 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:38.804+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:38.861 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.1. 2024-09-17T23:31:38.876 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:38.877 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:38.877 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:38.877 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:38.877 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:38.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:31:38.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:38.928 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:38.926+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:39.008 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:39.006+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:39.037 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:39.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 159.56296421100001, 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:39.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:39.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.705128398, 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:39.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.704436105, 2024-09-17T23:31:39.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:39.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.704424524, 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:39.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.703897277, 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:39.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.703863294, 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:39.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.703552702, 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:39.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:39.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.703526568, 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:39.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:39.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.703028661, 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:39.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11994 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4415488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.702427571, 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050474333000000003, 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11994 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331976+0000", 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332255+0000", 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027849100000000001 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332259+0000", 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332268+0000", 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3090000000000001e-06 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332275+0000", 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382276+0000", 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050001212000000003 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382366+0000", 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9693999999999999e-05 2024-09-17T23:31:39.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382451+0000", 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.454e-05 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.701878936, 2024-09-17T23:31:39.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:39.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.701671338, 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:39.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:39.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.69205543699999, 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:39.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.69204520300001, 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.692035336, 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.692025188, 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.69155532000001, 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.691533204, 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.505859339, 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.132262711999999, 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:31:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:31:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:31:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.137 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:39.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:39.139 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:39.137+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:39.240 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:39.238+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:39.247 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:39.247 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:39.247 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:39.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:39.340 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:39.339+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:39.403 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:39.404 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:39.404 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:39.404 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:39.404 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:39.412 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:39.453 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:39.451+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:39.542 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:39.541+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.522788804, 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.522615864, 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.52257599399999, 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.228071767, 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.980370218, 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.970437952, 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.957210903, 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.416681746999998, 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.416660356999998, 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.416643227999998, 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.416625945999996, 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.416608917000005, 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:39.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:39.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:39.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.416576043000006, 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:39.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.415988337000002, 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:39.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:39.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.394196195999996, 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:39.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:39.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:39.642 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.393659008, 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:39.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:39.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.393172891999995, 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:39.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.321030289999996, 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:39.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:39.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:39.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.321008479, 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:39.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.311565608999999, 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:39.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:39.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:39.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:39.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:39.669 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:39.670 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:39.671 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:39.668+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:39.770 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:39.769+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:39.791 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:39.791 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:39.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:39.862 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:31:39.871 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:39.870+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:39.912 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:39.912 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:39.912 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:39.912 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:39.912 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:39.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:39.972 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:39.970+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:40.029 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:40.029 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.15186481399999, 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.030 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:40.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:40.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:40.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:40.036 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.136239684, 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.037 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:40.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:40.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.13620202800001, 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:40.044 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:40.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.13618459700001, 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:40.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.054 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.13614966599999, 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.055 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.136118246, 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:40.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.13610162000001, 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.066 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:40.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.13607992600001, 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.071 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.072 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:40.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.13606820800001, 2024-09-17T23:31:40.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:40.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:40.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.176929689000005, 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:40.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:40.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:40.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:40.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:40.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:40.089 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.172974525000001, 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:40.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.094 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.172933866999998, 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:40.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:40.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.172674442000002, 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:40.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:40.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.172638219999996, 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:40.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:40.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.172605924999999, 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:40.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.172451113999998, 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:40.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:40.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.172164793999997, 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:40.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:40.127 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.172096132999997, 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:40.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:40.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 74.172054158999998, 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:40.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 19.172270136000002, 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:40.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:40.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:40.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:40.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.144 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.144 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.144 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:40.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:40.146 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:40.143+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:40.246 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:40.244+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:40.294 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:40.294 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:40.294 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:40.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:40.347 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:40.345+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:40.447 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:40.446+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:40.450 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:40.450 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:40.450 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:40.450 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:40.450 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:40.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:40.548 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:40.546+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.567 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.070194263, 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:40.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:40.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:40.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:40.573 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.07017438299999, 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:40.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:40.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:40.578 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.070152845, 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:40.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:40.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:40.582 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.068260131, 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:40.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.587 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.06824790100001, 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:40.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.068237332, 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:40.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:40.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.068227444, 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:40.596 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:40.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.601 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.068217162, 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:40.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.06820127500001, 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:40.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:40.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.529275958, 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:40.611 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.527206572, 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:40.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.19533920400001, 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:40.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:40.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.710374525999995, 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:40.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:40.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:40.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.710342155999996, 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:40.635 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.640 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.710049779000002, 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.641 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:40.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:40.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.704225640999994, 2024-09-17T23:31:40.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:40.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:40.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:40.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.704184138000002, 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:40.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.661445870999998, 2024-09-17T23:31:40.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:40.658 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.659 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:40.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 79.661343529999996, 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.665 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:40.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:40.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:40.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:40.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:40.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:40.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.6810296280000001, 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:40.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:31:40.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:31:40.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:40.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:40.677 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:40.679 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:40.679 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:40.677+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:40.780 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:40.778+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:40.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:40.827 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:40.827 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:40.827 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:40.827 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:40.836 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:40.881 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:40.879+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:40.956 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:40.965 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:40.965 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:40.965 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:40.965 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:40.966 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:40.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:40.978 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:40.997 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:41.019 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:41.017+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:41.019 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:41.020 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:41.039 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:41.039 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:41.059 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:41.092 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.093 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:41.115 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:41.113+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:41.183 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:41.181+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:41.228 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.228 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:41.284 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:41.282+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:41.385 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:41.383+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:41.386 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.386 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:41.485 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:41.483+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:41.505 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.505 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:41.587 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:41.585+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:41.626 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.626 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:41.688 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:41.686+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:41.784 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.784 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.788 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:41.786+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:41.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:41.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:31:41.889 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:41.887+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:41.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:41.990 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:41.988+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:42.081 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.082 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:42.092 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:42.089+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:42.192 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:42.190+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:42.198 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.199 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:42.293 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:42.291+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:42.318 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.318 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:42.394 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:42.392+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:42.444 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.444 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:42.496 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:42.494+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:42.597 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:42.595+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:42.604 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.604 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.613 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:42.697 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:42.696+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:42.724 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.724 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.733 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:42.799 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:42.797+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:42.884 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:42.884 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:42.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:42.937 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:42.935+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:42.942 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:31:43.003 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:43.002+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:43.006 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:43.007 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:43.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:43.073 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:43.101 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:43.099+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:43.162 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:43.162 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:43.170 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:43.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:43.196 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:43.212 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:43.210+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:43.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:43.213 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:43.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:43.229 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:43.246 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:43.246 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:43.263 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:43.263 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:43.280 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:43.281 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:43.302 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:43.300+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:43.319 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:43.319 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:43.319 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:43.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:43.412 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:43.410+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:43.475 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:43.476 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:43.476 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:43.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:43.525 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:43.523+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:43.604 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:43.603+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:43.633 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:43.633 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:43.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:43.633 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:43.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1209 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 15597568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.425461974000001, 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044407630000000003, 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1209 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.634 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204729+0000", 2024-09-17T23:31:43.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.077e-06 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204736+0000", 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7889999999999997e-06 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204747+0000", 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0793e-05 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204754+0000", 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3379999999999999e-06 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204820+0000", 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6235000000000005e-05 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204911+0000", 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1024000000000001e-05 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.205182+0000", 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000270134 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.208700+0000", 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035188810000000002 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209111+0000", 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00041066099999999999 2024-09-17T23:31:43.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209144+0000", 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2653000000000001e-05 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209169+0000", 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5177999999999998e-05 2024-09-17T23:31:43.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1629 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.250738171999998, 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038611639999999998, 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1629 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:43.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379452+0000", 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379453+0000", 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1429999999999999e-06 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379457+0000", 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0969999999999999e-06 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379469+0000", 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1868999999999999e-05 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379476+0000", 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2400000000000001e-06 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379533+0000", 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6666999999999998e-05 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379584+0000", 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1057000000000003e-05 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.379647+0000", 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2778000000000001e-05 2024-09-17T23:31:43.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380452+0000", 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080491199999999997 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383214+0000", 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002761922 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383272+0000", 2024-09-17T23:31:43.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8184999999999999e-05 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.383313+0000", 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1294000000000001e-05 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:43.646 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.250122761, 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:43.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1643 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14811136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.242488622, 2024-09-17T23:31:43.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042815140000000002, 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1643 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:43.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387703+0000", 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.308e-06 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387708+0000", 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.865e-06 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387723+0000", 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.556e-05 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387732+0000", 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5099999999999998e-06 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387795+0000", 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2929999999999995e-05 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387897+0000", 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102815 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388033+0000", 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013546899999999999 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388921+0000", 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088850099999999998 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391906+0000", 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002984202 2024-09-17T23:31:43.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391944+0000", 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8565000000000001e-05 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391983+0000", 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8788999999999997e-05 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21422 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.649440304999999, 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044109429999999996, 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21422 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980751+0000", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980755+0000", 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980768+0000", 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2862e-05 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980776+0000", 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8359999999999992e-06 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980833+0000", 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6749000000000002e-05 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980879+0000", 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6050999999999997e-05 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980977+0000", 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8488999999999998e-05 2024-09-17T23:31:43.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982120+0000", 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001143257 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985096+0000", 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00297569 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985133+0000", 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7134000000000001e-05 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985161+0000", 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7514000000000001e-05 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:43.663 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.647871709, 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:43.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:43.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 25.547239072, 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:43.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:43.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:43.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22086 3.0 3:04a20ea7:::rbd_data.1152c10d4b9c.000000000000000b:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.399252502, 2024-09-17T23:31:43.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0043112539999999996, 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22086 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230939+0000", 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3370000000000001e-06 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230945+0000", 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.37e-06 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230966+0000", 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0492999999999998e-05 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230991+0000", 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5463e-05 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232196+0000", 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012043069999999999 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232366+0000", 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017094000000000001 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232576+0000", 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020904100000000001 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233937+0000", 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013611140000000001 2024-09-17T23:31:43.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235062+0000", 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001125175 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235117+0000", 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4976999999999999e-05 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235249+0000", 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000132037 2024-09-17T23:31:43.683 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22095 3.0 3:03401d0b:::rbd_data.1152c10d4b9c.000000000000001d:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.396237601999999, 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003963388, 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.684 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22095 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233952+0000", 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233954+0000", 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.181e-06 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233959+0000", 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4569999999999996e-06 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233971+0000", 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1924e-05 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233976+0000", 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4310000000000002e-06 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234018+0000", 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1050000000000002e-05 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234197+0000", 2024-09-17T23:31:43.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017971 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.234374+0000", 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017669300000000001 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235543+0000", 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011695060000000001 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237714+0000", 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021710039999999998 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237765+0000", 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0853999999999998e-05 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237916+0000", 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015057800000000001 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.689 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.394711426000001, 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:31:43.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:31:43.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.394574187, 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.696 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:31:43.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:43.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:43.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 21.389417027, 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:31:43.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:31:43.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22907 3.0 3:07f1f9a0:::rbd_data.11524e94aa61.0000000000000007:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.654363042, 2024-09-17T23:31:43.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053158520000000003, 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22907 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975829+0000", 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975838+0000", 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3260000000000002e-06 2024-09-17T23:31:43.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975859+0000", 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0378999999999999e-05 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975870+0000", 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1488000000000001e-05 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975948+0000", 2024-09-17T23:31:43.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199000000000001e-05 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976110+0000", 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000161977 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976234+0000", 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012316599999999999 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.977277+0000", 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010435679999999999 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981028+0000", 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037512130000000002 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981092+0000", 2024-09-17T23:31:43.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3468999999999998e-05 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981143+0000", 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1017999999999998e-05 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22969 3.17 3:e879c454:::rbd_header.1152324fa224:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.741193600999999, 2024-09-17T23:31:43.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058282270000000001, 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22969 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:43.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888998+0000", 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889005+0000", 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000004e-06 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889017+0000", 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2478e-05 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889029+0000", 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.168e-05 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889311+0000", 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028192400000000002 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889435+0000", 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000124046 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889886+0000", 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00045141000000000002 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.892147+0000", 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002260813 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894710+0000", 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025627710000000001 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894768+0000", 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8059000000000001e-05 2024-09-17T23:31:43.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894825+0000", 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6986e-05 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23066 3.1f 3:fea969d6:::rbd_data.1152324fa224.0000000000000007:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.657557583999999, 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048693310000000002, 2024-09-17T23:31:43.719 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23066 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.569999999999999e-07 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972638+0000", 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8040000000000001e-06 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972649+0000", 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0882e-05 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972656+0000", 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:43.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973702+0000", 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010456809999999999 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973830+0000", 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128071 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973960+0000", 2024-09-17T23:31:43.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013065799999999999 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.975260+0000", 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012997729999999999 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977401+0000", 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021405030000000002 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977457+0000", 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168999999999997e-05 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977502+0000", 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4919000000000001e-05 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23135 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'391, mlcod=0'0)", 2024-09-17T23:31:43.725 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:43.726 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.5269817349999997, 2024-09-17T23:31:43.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053043279999999996, 2024-09-17T23:31:43.726 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.726 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103209+0000", 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:31:43.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103215+0000", 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999998e-06 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103222+0000", 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4919999999999996e-06 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103231+0000", 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6049999999999998e-06 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103277+0000", 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6641999999999998e-05 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103302+0000", 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4601e-05 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108442+0000", 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051404129999999999 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108513+0000", 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0192999999999997e-05 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23188 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'395, mlcod=23'1)", 2024-09-17T23:31:43.731 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.161308+0000", 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4688821060000006, 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003311109, 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161308+0000", 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161308+0000", 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161309+0000", 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9800000000000003e-07 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161315+0000", 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7239999999999997e-06 2024-09-17T23:31:43.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161327+0000", 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2218000000000001e-05 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161333+0000", 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.5100000000000004e-06 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161374+0000", 2024-09-17T23:31:43.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1196999999999999e-05 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.161391+0000", 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6620999999999999e-05 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.164576+0000", 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003185289 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.164619+0000", 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2752e-05 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23195 3.1c e131/130 3:3831950c:::rbd_directory:head v 131'414, mlcod=0'0)", 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.4450760349999996, 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047496070000000003, 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185115+0000", 2024-09-17T23:31:43.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1900000000000003e-07 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185119+0000", 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9230000000000001e-06 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185128+0000", 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0530000000000007e-06 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185132+0000", 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3889999999999998e-06 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185180+0000", 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8059000000000002e-05 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185205+0000", 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4865999999999999e-05 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.189830+0000", 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046256379999999996 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.189864+0000", 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3160000000000001e-05 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23232 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'398, mlcod=23'1)", 2024-09-17T23:31:43.740 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.3984448450000002, 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048635629999999996, 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231746+0000", 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999996e-07 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231752+0000", 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7790000000000001e-06 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231760+0000", 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6279999999999994e-06 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231765+0000", 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8919999999999999e-06 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231809+0000", 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3544000000000002e-05 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231837+0000", 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7815999999999999e-05 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236575+0000", 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047384030000000004 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236609+0000", 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3896999999999997e-05 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.744 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23275 3.1 e132/130 3:86cbfba3:::rbd_header.11528a3e36a5:head v 132'803, mlcod=23'42)", 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.861153+0000", 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.7690370990000002, 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044164490000000002, 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:43.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861153+0000", 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861153+0000", 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861154+0000", 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1600000000000001e-07 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:43.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861160+0000", 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3790000000000003e-06 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861168+0000", 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2460000000000003e-06 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:43.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861173+0000", 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.95e-06 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861225+0000", 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1730000000000001e-05 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861249+0000", 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4307e-05 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.865288+0000", 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0040388239999999999 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.865569+0000", 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028149700000000002 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:43.749 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:43.751 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:43.749+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:43.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:43.852 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:43.851+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:43.868 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:43.868 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:43.868 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:43.868 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:43.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:43.879 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:31:43.880 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:43.880 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T23:31:43.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:31:43.953 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:43.951+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:43.990 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:43.990 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:43.990 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:43.990 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:43.990 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:43.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:44.053 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:44.051+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.270028744000001, 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:44.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:44.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:44.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.260456645000005, 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:44.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.256926139000001, 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:44.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:44.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.130 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.255431432999998, 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:44.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:44.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:44.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.255248514000002, 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.135 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:44.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.255135013, 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.255110905999999, 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:44.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:44.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.255036575000005, 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:44.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.255013026, 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:44.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.251841825, 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:44.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.247433489000002, 2024-09-17T23:31:44.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:44.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:44.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.220161382000001, 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:44.169 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:44.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.210979230000007, 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.210947736999998, 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:44.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:44.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.210922045000004, 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:44.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:44.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:44.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.210777492999995, 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:44.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.209209376000004, 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:44.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:44.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.209012028999993, 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:44.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.208985713999994, 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.208957370999997, 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:44.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:44.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:44.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.217 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:44.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:44.219 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:44.217+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 140, 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:31:43.937923+0000", 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:31:11.038388+0000", 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:31:38.858088+0000", 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-17T23:31:44.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:31:44.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:31:44.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:31:44.225 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:31:44.226 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:31:44.227 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:31:44.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:31:44.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.230 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:31:44.231 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:31:44.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:31:44.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:31:44.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:31:44.235 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 3.7300000190734863, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 3.7300000190734863, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.5, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.5, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:31:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:31:44.238 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "138", 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 45, 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 138, 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:31:44.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:31:44.240 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:31:44.241 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.059999942779541, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.059999942779541, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:31:44.242 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.77099609375, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:31:44.243 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 139, 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:31:44.244 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:31:44.245 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:31:44.247 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:31:44.248 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:31:44.249 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 138, 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:31:44.250 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:31:44.251 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:31:44.252 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:31:44.253 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:31:44.254 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:31:44.259 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.7953948974609375, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:31:44.260 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 139, 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.263 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.89776611328125, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 139, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:31:44.266 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:31:44.271 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.1843414306640625, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 11, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 93, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 107, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 136, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 94, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.272 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1359318569", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1359318569", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1359318569", 2024-09-17T23:31:44.276 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1359318569", 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:31:44.277 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 55177, 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:31:44.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:31:44.279 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:31:44.280 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 94 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.281 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.12", 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: 2, 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [ 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.16", 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:31:44.283 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:31:44.284 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 32, 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 11 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:31:44.285 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:31:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:31:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:31:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:31:44.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.286 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:44.287 INFO:tasks.thrashosds.thrasher:j is {'epoch': 140, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:31:43.937923+0000', 'last_up_change': '2024-09-17T23:31:11.038388+0000', 'last_in_change': '2024-09-17T23:31:38.858088+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.7300000190734863, 'score_stable': 3.7300000190734863, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 138, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.059999942779541, 'score_stable': 2.059999942779541, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 0.77099609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 139, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 0.7953948974609375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 139, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 0.89776611328125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 139, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1843414306640625, 'last_clean_begin': 11, 'last_clean_end': 93, 'up_from': 107, 'up_thru': 136, 'down_at': 94, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1359318569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1359318569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1359318569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1359318569}]}, 'public_addr': '172.21.15.37:6809/1359318569', 'cluster_addr': '172.21.15.37:6811/1359318569', 'heartbeat_back_addr': '172.21.15.37:6815/1359318569', 'heartbeat_front_addr': '172.21.15.37:6813/1359318569', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 55177, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 94}], 'pg_upmap': [{'pgid': '3.12', 'osds': [2, 1]}, {'pgid': '3.17', 'osds': [5, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.16', 'osds': [5, 1]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 32, 'length': 11}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:31:44.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:31:44.319 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:44.317+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:44.394 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:44.394 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:44.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:44.446 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:44.444+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:44.512 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:44.512 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:44.512 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:44.512 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:44.513 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:44.520 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:44.517+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:44.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:44.621 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:44.622 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:31:44.623 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:44.618+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-17T23:31:44.388880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1078701816,"num_objects":1332,"num_object_clones":343,"num_object_copies":2664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1332,"num_whiteouts":560,"num_read":306989,"num_read_kb":1321558,"num_write":18825,"num_write_kb":483007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6039,"num_bytes_recovered":241239163,"num_keys_recovered":1110,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16994,"ondisk_log_size":16994,"up":81,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":263556,"kb_used_data":30560,"kb_used_omap":24,"kb_used_meta":232871,"kb_avail":377223804,"statfs":{"total":386547056640,"available":386277175296,"internally_reserved":0,"allocated":31293440,"data_stored":174334840,"data_compressed":8921549,"data_compressed_allocated":18440192,"data_compressed_original":168091136,"omap_allocated":24578,"internal_metadata":238460926},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8301143,"num_objects":-16,"num_object_clones":-4,"num_object_copies":-32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-16,"num_whiteouts":-4,"num_read":227,"num_read_kb":197,"num_write":48,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1227,"num_bytes_recovered":34774336,"num_keys_recovered":140,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000675"},"pg_stats":[{"pgid":"3.1f","version":"132'403","reported_seq":5657,"reported_epoch":139,"state":"peering","last_fresh":"2024-09-17T23:31:42.704994+0000","last_change":"2024-09-17T23:31:43.397286+0000","last_active":"2024-09-17T23:31:43.397286+0000","last_peered":"2024-09-17T23:31:42.704994+0000","last_clean":"2024-09-17T23:31:35.864920+0000","last_became_active":"2024-09-17T23:31:40.901489+0000","last_became_peered":"2024-09-17T23:31:40.901489+0000","last_unstale":"2024-09-17T23:31:42.704994+0000","last_undegraded":"2024-09-17T23:31:42.704994+0000","last_fullsized":"2024-09-17T23:31:42.704994+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0012858920000000001,"stat_sum":{"num_bytes":32570420,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":7434,"num_read_kb":66454,"num_write":451,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":21604086,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"9"}]},{"pgid":"3.1e","version":"137'485","reported_seq":8693,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.950237+0000","last_change":"2024-09-17T23:31:42.950237+0000","last_active":"2024-09-17T23:31:42.950237+0000","last_peered":"2024-09-17T23:31:42.950237+0000","last_clean":"2024-09-17T23:31:42.950237+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:31:42.950237+0000","last_undegraded":"2024-09-17T23:31:42.950237+0000","last_fullsized":"2024-09-17T23:31:42.950237+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:53.590264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091699299999999995,"stat_sum":{"num_bytes":82797394,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":18,"num_read":9080,"num_read_kb":54882,"num_write":505,"num_write_kb":19035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.1d","version":"132'313","reported_seq":5613,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.948886+0000","last_change":"2024-09-17T23:31:42.948886+0000","last_active":"2024-09-17T23:31:42.948886+0000","last_peered":"2024-09-17T23:31:42.948886+0000","last_clean":"2024-09-17T23:31:42.948886+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:31:42.948886+0000","last_undegraded":"2024-09-17T23:31:42.948886+0000","last_fullsized":"2024-09-17T23:31:42.948886+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:48:36.319103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082175900000000003,"stat_sum":{"num_bytes":18891814,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":4635,"num_read_kb":41057,"num_write":300,"num_write_kb":14196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.1c","version":"137'418","reported_seq":8992,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.950130+0000","last_change":"2024-09-17T23:31:42.949710+0000","last_active":"2024-09-17T23:31:42.950130+0000","last_peered":"2024-09-17T23:31:42.950130+0000","last_clean":"2024-09-17T23:31:42.950130+0000","last_became_active":"2024-09-17T23:31:42.946855+0000","last_became_peered":"2024-09-17T23:31:42.946855+0000","last_unstale":"2024-09-17T23:31:42.950130+0000","last_undegraded":"2024-09-17T23:31:42.950130+0000","last_fullsized":"2024-09-17T23:31:42.950130+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:24.374014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001472328,"stat_sum":{"num_bytes":33247760,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":8395,"num_read_kb":40143,"num_write":498,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":14561502,"num_keys_recovered":350,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.1b","version":"134'402","reported_seq":19224,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940110+0000","last_change":"2024-09-17T23:31:41.945954+0000","last_active":"2024-09-17T23:31:42.940110+0000","last_peered":"2024-09-17T23:31:42.940110+0000","last_clean":"2024-09-17T23:31:42.940110+0000","last_became_active":"2024-09-17T23:31:37.864076+0000","last_became_peered":"2024-09-17T23:31:37.864076+0000","last_unstale":"2024-09-17T23:31:42.940110+0000","last_undegraded":"2024-09-17T23:31:42.940110+0000","last_fullsized":"2024-09-17T23:31:42.940110+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:46.155431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016323220000000001,"stat_sum":{"num_bytes":67564347,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":18861,"num_read_kb":58408,"num_write":451,"num_write_kb":23018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":2836795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.1a","version":"134'847","reported_seq":7265,"reported_epoch":139,"state":"peering","last_fresh":"2024-09-17T23:31:42.451621+0000","last_change":"2024-09-17T23:31:43.397239+0000","last_active":"2024-09-17T23:31:43.397239+0000","last_peered":"2024-09-17T23:31:42.451621+0000","last_clean":"2024-09-17T23:31:37.987403+0000","last_became_active":"2024-09-17T23:31:40.901370+0000","last_became_peered":"2024-09-17T23:31:40.901370+0000","last_unstale":"2024-09-17T23:31:42.451621+0000","last_undegraded":"2024-09-17T23:31:42.451621+0000","last_fullsized":"2024-09-17T23:31:42.451621+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019548875,"stat_sum":{"num_bytes":33840829,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":11868,"num_read_kb":30774,"num_write":1188,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"22","length":"9"}]},{"pgid":"3.19","version":"134'787","reported_seq":6427,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.948466+0000","last_change":"2024-09-17T23:31:42.948466+0000","last_active":"2024-09-17T23:31:42.948466+0000","last_peered":"2024-09-17T23:31:42.948466+0000","last_clean":"2024-09-17T23:31:42.948466+0000","last_became_active":"2024-09-17T23:31:42.945986+0000","last_became_peered":"2024-09-17T23:31:42.945986+0000","last_unstale":"2024-09-17T23:31:42.948466+0000","last_undegraded":"2024-09-17T23:31:42.948466+0000","last_fullsized":"2024-09-17T23:31:42.948466+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:06.363649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080910200000000002,"stat_sum":{"num_bytes":26913876,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":8527,"num_read_kb":17537,"num_write":907,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":4472153,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.18","version":"132'430","reported_seq":4191,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940858+0000","last_change":"2024-09-17T23:31:41.945933+0000","last_active":"2024-09-17T23:31:42.940858+0000","last_peered":"2024-09-17T23:31:42.940858+0000","last_clean":"2024-09-17T23:31:42.940858+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:31:42.940858+0000","last_undegraded":"2024-09-17T23:31:42.940858+0000","last_fullsized":"2024-09-17T23:31:42.940858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:35.961928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018876679999999999,"stat_sum":{"num_bytes":18727476,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":19,"num_read":4986,"num_read_kb":22191,"num_write":432,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.17","version":"128'590","reported_seq":7826,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T23:31:39.868527+0000","last_change":"2024-09-17T23:31:38.863520+0000","last_active":"2024-09-17T23:31:39.868527+0000","last_peered":"2024-09-17T23:31:39.868527+0000","last_clean":"2024-09-17T23:31:39.868527+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:31:39.868527+0000","last_undegraded":"2024-09-17T23:31:39.868527+0000","last_fullsized":"2024-09-17T23:31:39.868527+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:06:19.958774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602268,"stat_sum":{"num_bytes":37074050,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":10656,"num_read_kb":28724,"num_write":513,"num_write_kb":13263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"21","length":"a"}]},{"pgid":"3.16","version":"133'343","reported_seq":4188,"reported_epoch":136,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:39.869082+0000","last_change":"2024-09-17T23:31:39.869082+0000","last_active":"2024-09-17T23:31:38.862886+0000","last_peered":"2024-09-17T23:31:37.863284+0000","last_clean":"2024-09-17T23:31:37.863284+0000","last_became_active":"2024-09-17T23:31:36.864284+0000","last_became_peered":"2024-09-17T23:31:36.864284+0000","last_unstale":"2024-09-17T23:31:39.869082+0000","last_undegraded":"2024-09-17T23:31:39.869082+0000","last_fullsized":"2024-09-17T23:31:39.869082+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.013226596,"stat_sum":{"num_bytes":15285808,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":4431,"num_read_kb":31183,"num_write":358,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5911184,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"9"}]},{"pgid":"3.15","version":"137'564","reported_seq":12669,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942795+0000","last_change":"2024-09-17T23:31:41.941087+0000","last_active":"2024-09-17T23:31:42.942795+0000","last_peered":"2024-09-17T23:31:42.942795+0000","last_clean":"2024-09-17T23:31:42.942795+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:31:42.942795+0000","last_undegraded":"2024-09-17T23:31:42.942795+0000","last_fullsized":"2024-09-17T23:31:42.942795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:05.629874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040779999999999999,"stat_sum":{"num_bytes":32097361,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":14488,"num_read_kb":31548,"num_write":600,"num_write_kb":6540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.14","version":"126'518","reported_seq":7772,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942556+0000","last_change":"2024-09-17T23:31:41.940233+0000","last_active":"2024-09-17T23:31:42.942556+0000","last_peered":"2024-09-17T23:31:42.942556+0000","last_clean":"2024-09-17T23:31:42.942556+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:31:42.942556+0000","last_undegraded":"2024-09-17T23:31:42.942556+0000","last_fullsized":"2024-09-17T23:31:42.942556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:48.875742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083373200000000001,"stat_sum":{"num_bytes":21155425,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":8401,"num_read_kb":33555,"num_write":448,"num_write_kb":18362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":212,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942892+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:31:42.942892+0000","last_peered":"2024-09-17T23:31:42.942892+0000","last_clean":"2024-09-17T23:31:42.942892+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:31:42.942892+0000","last_undegraded":"2024-09-17T23:31:42.942892+0000","last_fullsized":"2024-09-17T23:31:42.942892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"137'589","reported_seq":4945,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943726+0000","last_change":"2024-09-17T23:31:41.942553+0000","last_active":"2024-09-17T23:31:42.943726+0000","last_peered":"2024-09-17T23:31:42.943726+0000","last_clean":"2024-09-17T23:31:42.943726+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:31:42.943726+0000","last_undegraded":"2024-09-17T23:31:42.943726+0000","last_fullsized":"2024-09-17T23:31:42.943726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:53.303312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071435300000000004,"stat_sum":{"num_bytes":13427287,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":5974,"num_read_kb":22618,"num_write":870,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":262,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.944015+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:31:42.944015+0000","last_peered":"2024-09-17T23:31:42.944015+0000","last_clean":"2024-09-17T23:31:42.944015+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:31:42.944015+0000","last_undegraded":"2024-09-17T23:31:42.944015+0000","last_fullsized":"2024-09-17T23:31:42.944015+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"134'407","reported_seq":12382,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.944285+0000","last_change":"2024-09-17T23:31:41.942789+0000","last_active":"2024-09-17T23:31:42.944285+0000","last_peered":"2024-09-17T23:31:42.944285+0000","last_clean":"2024-09-17T23:31:42.944285+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:31:42.944285+0000","last_undegraded":"2024-09-17T23:31:42.944285+0000","last_fullsized":"2024-09-17T23:31:42.944285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:19:35.823842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054858800000000003,"stat_sum":{"num_bytes":19786283,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":13,"num_read":13024,"num_read_kb":60475,"num_write":416,"num_write_kb":30569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.5","version":"0'0","reported_seq":222,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943918+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:31:42.943918+0000","last_peered":"2024-09-17T23:31:42.943918+0000","last_clean":"2024-09-17T23:31:42.943918+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:31:42.943918+0000","last_undegraded":"2024-09-17T23:31:42.943918+0000","last_fullsized":"2024-09-17T23:31:42.943918+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"137'375","reported_seq":7140,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943819+0000","last_change":"2024-09-17T23:31:41.942342+0000","last_active":"2024-09-17T23:31:42.943819+0000","last_peered":"2024-09-17T23:31:42.943819+0000","last_clean":"2024-09-17T23:31:42.943819+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:31:42.943819+0000","last_undegraded":"2024-09-17T23:31:42.943819+0000","last_fullsized":"2024-09-17T23:31:42.943819+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:43.305144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059104399999999999,"stat_sum":{"num_bytes":18939983,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":24,"num_read":4591,"num_read_kb":19708,"num_write":430,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943035+0000","last_change":"2024-09-17T23:31:33.918893+0000","last_active":"2024-09-17T23:31:42.943035+0000","last_peered":"2024-09-17T23:31:42.943035+0000","last_clean":"2024-09-17T23:31:42.943035+0000","last_became_active":"2024-09-17T23:31:33.918555+0000","last_became_peered":"2024-09-17T23:31:33.918555+0000","last_unstale":"2024-09-17T23:31:42.943035+0000","last_undegraded":"2024-09-17T23:31:42.943035+0000","last_fullsized":"2024-09-17T23:31:42.943035+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:01:56.872633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"128'358","reported_seq":10379,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940488+0000","last_change":"2024-09-17T23:31:41.946552+0000","last_active":"2024-09-17T23:31:42.940488+0000","last_peered":"2024-09-17T23:31:42.940488+0000","last_clean":"2024-09-17T23:31:42.940488+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:31:42.940488+0000","last_undegraded":"2024-09-17T23:31:42.940488+0000","last_fullsized":"2024-09-17T23:31:42.940488+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:57.408399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017503,"stat_sum":{"num_bytes":23530074,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":10189,"num_read_kb":27179,"num_write":424,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.3","version":"138'502","reported_seq":7128,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942719+0000","last_change":"2024-09-17T23:31:41.940754+0000","last_active":"2024-09-17T23:31:42.942719+0000","last_peered":"2024-09-17T23:31:42.942719+0000","last_clean":"2024-09-17T23:31:42.942719+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:31:42.942719+0000","last_undegraded":"2024-09-17T23:31:42.942719+0000","last_fullsized":"2024-09-17T23:31:42.942719+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:04:26.875116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010678510000000001,"stat_sum":{"num_bytes":29375012,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":29,"num_read":7187,"num_read_kb":29652,"num_write":533,"num_write_kb":9119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.2","version":"18'2","reported_seq":307,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942436+0000","last_change":"2024-09-17T23:31:39.878887+0000","last_active":"2024-09-17T23:31:42.942436+0000","last_peered":"2024-09-17T23:31:42.942436+0000","last_clean":"2024-09-17T23:31:42.942436+0000","last_became_active":"2024-09-17T23:31:39.873374+0000","last_became_peered":"2024-09-17T23:31:39.873374+0000","last_unstale":"2024-09-17T23:31:42.942436+0000","last_undegraded":"2024-09-17T23:31:42.942436+0000","last_fullsized":"2024-09-17T23:31:42.942436+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:49.977116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"137'625","reported_seq":11893,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.948493+0000","last_change":"2024-09-17T23:31:42.948493+0000","last_active":"2024-09-17T23:31:42.948493+0000","last_peered":"2024-09-17T23:31:42.948493+0000","last_clean":"2024-09-17T23:31:42.948493+0000","last_became_active":"2024-09-17T23:31:42.945710+0000","last_became_peered":"2024-09-17T23:31:42.945710+0000","last_unstale":"2024-09-17T23:31:42.948493+0000","last_undegraded":"2024-09-17T23:31:42.948493+0000","last_fullsized":"2024-09-17T23:31:42.948493+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:33.470574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099364500000000003,"stat_sum":{"num_bytes":15584385,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":12748,"num_read_kb":56367,"num_write":699,"num_write_kb":10223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":11442867,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":267,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940462+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:31:42.940462+0000","last_peered":"2024-09-17T23:31:42.940462+0000","last_clean":"2024-09-17T23:31:42.940462+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:31:42.940462+0000","last_undegraded":"2024-09-17T23:31:42.940462+0000","last_fullsized":"2024-09-17T23:31:42.940462+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"134'823","reported_seq":8812,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942285+0000","last_change":"2024-09-17T23:31:41.939838+0000","last_active":"2024-09-17T23:31:42.942285+0000","last_peered":"2024-09-17T23:31:42.942285+0000","last_clean":"2024-09-17T23:31:42.942285+0000","last_became_active":"2024-09-17T23:31:37.868856+0000","last_became_peered":"2024-09-17T23:31:37.868856+0000","last_unstale":"2024-09-17T23:31:42.942285+0000","last_undegraded":"2024-09-17T23:31:42.942285+0000","last_fullsized":"2024-09-17T23:31:42.942285+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:02:38.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051005199999999999,"stat_sum":{"num_bytes":33188027,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":11282,"num_read_kb":15888,"num_write":664,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":4479025,"num_keys_recovered":150,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":130,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943386+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:31:42.943386+0000","last_peered":"2024-09-17T23:31:42.943386+0000","last_clean":"2024-09-17T23:31:42.943386+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:31:42.943386+0000","last_undegraded":"2024-09-17T23:31:42.943386+0000","last_fullsized":"2024-09-17T23:31:42.943386+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"134'624","reported_seq":10768,"reported_epoch":138,"state":"peering","last_fresh":"2024-09-17T23:31:42.255270+0000","last_change":"2024-09-17T23:31:42.940747+0000","last_active":"2024-09-17T23:31:42.940747+0000","last_peered":"2024-09-17T23:31:42.255270+0000","last_clean":"2024-09-17T23:31:37.887805+0000","last_became_active":"2024-09-17T23:31:40.901369+0000","last_became_peered":"2024-09-17T23:31:40.901369+0000","last_unstale":"2024-09-17T23:31:42.255270+0000","last_undegraded":"2024-09-17T23:31:42.255270+0000","last_fullsized":"2024-09-17T23:31:40.866940+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015073197,"stat_sum":{"num_bytes":56002697,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":10977,"num_read_kb":79105,"num_write":720,"num_write_kb":59930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15860611,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"9"}]},{"pgid":"2.3","version":"16'1","reported_seq":185,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942803+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:31:42.942803+0000","last_peered":"2024-09-17T23:31:42.942803+0000","last_clean":"2024-09-17T23:31:42.942803+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:31:42.942803+0000","last_undegraded":"2024-09-17T23:31:42.942803+0000","last_fullsized":"2024-09-17T23:31:42.942803+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":351,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.947490+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:31:42.947490+0000","last_peered":"2024-09-17T23:31:42.947490+0000","last_clean":"2024-09-17T23:31:42.947490+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:31:42.947490+0000","last_undegraded":"2024-09-17T23:31:42.947490+0000","last_fullsized":"2024-09-17T23:31:42.947490+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"134'419","reported_seq":14577,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943556+0000","last_change":"2024-09-17T23:31:41.942248+0000","last_active":"2024-09-17T23:31:42.943556+0000","last_peered":"2024-09-17T23:31:42.943556+0000","last_clean":"2024-09-17T23:31:42.943556+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:31:42.943556+0000","last_undegraded":"2024-09-17T23:31:42.943556+0000","last_fullsized":"2024-09-17T23:31:42.943556+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:27.756584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051108299999999996,"stat_sum":{"num_bytes":42443021,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":15141,"num_read_kb":73921,"num_write":376,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.9","version":"128'668","reported_seq":11282,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940360+0000","last_change":"2024-09-17T23:31:41.948867+0000","last_active":"2024-09-17T23:31:42.940360+0000","last_peered":"2024-09-17T23:31:42.940360+0000","last_clean":"2024-09-17T23:31:42.940360+0000","last_became_active":"2024-09-17T23:31:36.863448+0000","last_became_peered":"2024-09-17T23:31:36.863448+0000","last_unstale":"2024-09-17T23:31:42.940360+0000","last_undegraded":"2024-09-17T23:31:42.940360+0000","last_fullsized":"2024-09-17T23:31:42.940360+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:22:14.213763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077353000000000003,"stat_sum":{"num_bytes":26111084,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":11108,"num_read_kb":39022,"num_write":795,"num_write_kb":25897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":1842284,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.a","version":"134'536","reported_seq":9517,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942324+0000","last_change":"2024-09-17T23:31:41.940225+0000","last_active":"2024-09-17T23:31:42.942324+0000","last_peered":"2024-09-17T23:31:42.942324+0000","last_clean":"2024-09-17T23:31:42.942324+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:31:42.942324+0000","last_undegraded":"2024-09-17T23:31:42.942324+0000","last_fullsized":"2024-09-17T23:31:42.942324+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:34:16.628679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081666300000000002,"stat_sum":{"num_bytes":20512573,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":10666,"num_read_kb":53049,"num_write":554,"num_write_kb":7229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.b","version":"138'851","reported_seq":10991,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940219+0000","last_change":"2024-09-17T23:31:41.946235+0000","last_active":"2024-09-17T23:31:42.940219+0000","last_peered":"2024-09-17T23:31:42.940219+0000","last_clean":"2024-09-17T23:31:42.940219+0000","last_became_active":"2024-09-17T23:31:37.863905+0000","last_became_peered":"2024-09-17T23:31:37.863905+0000","last_unstale":"2024-09-17T23:31:42.940219+0000","last_undegraded":"2024-09-17T23:31:42.940219+0000","last_fullsized":"2024-09-17T23:31:42.940219+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:50.880394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001889972,"stat_sum":{"num_bytes":37653062,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":14684,"num_read_kb":61189,"num_write":845,"num_write_kb":11825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2676806,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.c","version":"132'435","reported_seq":5234,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942610+0000","last_change":"2024-09-17T23:31:41.940699+0000","last_active":"2024-09-17T23:31:42.942610+0000","last_peered":"2024-09-17T23:31:42.942610+0000","last_clean":"2024-09-17T23:31:42.942610+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:31:42.942610+0000","last_undegraded":"2024-09-17T23:31:42.942610+0000","last_fullsized":"2024-09-17T23:31:42.942610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:41:14.855085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001079654,"stat_sum":{"num_bytes":21493439,"num_objects":49,"num_object_clones":16,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":22,"num_read":5958,"num_read_kb":36790,"num_write":446,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.d","version":"132'395","reported_seq":9572,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.952926+0000","last_change":"2024-09-17T23:31:42.952926+0000","last_active":"2024-09-17T23:31:42.952926+0000","last_peered":"2024-09-17T23:31:42.952926+0000","last_clean":"2024-09-17T23:31:42.952926+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:31:42.952926+0000","last_undegraded":"2024-09-17T23:31:42.952926+0000","last_fullsized":"2024-09-17T23:31:42.952926+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:26:21.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095911100000000001,"stat_sum":{"num_bytes":30469700,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":9068,"num_read_kb":42813,"num_write":454,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.e","version":"135'657","reported_seq":9840,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.940029+0000","last_change":"2024-09-17T23:31:41.946366+0000","last_active":"2024-09-17T23:31:42.940029+0000","last_peered":"2024-09-17T23:31:42.940029+0000","last_clean":"2024-09-17T23:31:42.940029+0000","last_became_active":"2024-09-17T23:31:38.869556+0000","last_became_peered":"2024-09-17T23:31:38.869556+0000","last_unstale":"2024-09-17T23:31:42.940029+0000","last_undegraded":"2024-09-17T23:31:42.940029+0000","last_fullsized":"2024-09-17T23:31:42.940029+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:42:23.941358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016742009999999999,"stat_sum":{"num_bytes":19644124,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":19,"num_read":10152,"num_read_kb":26207,"num_write":736,"num_write_kb":7184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":3804584,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.f","version":"137'351","reported_seq":4659,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.943854+0000","last_change":"2024-09-17T23:31:41.942563+0000","last_active":"2024-09-17T23:31:42.943854+0000","last_peered":"2024-09-17T23:31:42.943854+0000","last_clean":"2024-09-17T23:31:42.943854+0000","last_became_active":"2024-09-17T23:31:37.868909+0000","last_became_peered":"2024-09-17T23:31:37.868909+0000","last_unstale":"2024-09-17T23:31:42.943854+0000","last_undegraded":"2024-09-17T23:31:42.943854+0000","last_fullsized":"2024-09-17T23:31:42.943854+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:18.382400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000471712,"stat_sum":{"num_bytes":9040964,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4814,"num_read_kb":27403,"num_write":370,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":4115627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.10","version":"137'855","reported_seq":7588,"reported_epoch":139,"state":"creating+peering","last_fresh":"2024-09-17T23:31:42.903789+0000","last_change":"2024-09-17T23:31:43.397575+0000","last_active":"2024-09-17T23:31:43.397575+0000","last_peered":"2024-09-17T23:31:42.903789+0000","last_clean":"2024-09-17T23:31:37.970100+0000","last_became_active":"2024-09-17T23:31:40.907726+0000","last_became_peered":"2024-09-17T23:31:40.907726+0000","last_unstale":"2024-09-17T23:31:42.903789+0000","last_undegraded":"2024-09-17T23:31:42.903789+0000","last_fullsized":"2024-09-17T23:31:42.903789+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012531123,"stat_sum":{"num_bytes":23871113,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":9394,"num_read_kb":33877,"num_write":1177,"num_write_kb":24057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":7449751,"num_keys_recovered":56,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"22","length":"9"}]},{"pgid":"3.11","version":"132'343","reported_seq":8405,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.947388+0000","last_change":"2024-09-17T23:31:42.947388+0000","last_active":"2024-09-17T23:31:42.947388+0000","last_peered":"2024-09-17T23:31:42.947388+0000","last_clean":"2024-09-17T23:31:42.947388+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:31:42.947388+0000","last_undegraded":"2024-09-17T23:31:42.947388+0000","last_fullsized":"2024-09-17T23:31:42.947388+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:30.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022614570000000001,"stat_sum":{"num_bytes":79690752,"num_objects":40,"num_object_clones":13,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":16,"num_read":8068,"num_read_kb":46907,"num_write":325,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.12","version":"134'418","reported_seq":5658,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-17T23:31:42.942997+0000","last_change":"2024-09-17T23:31:41.939848+0000","last_active":"2024-09-17T23:31:42.942997+0000","last_peered":"2024-09-17T23:31:42.942997+0000","last_clean":"2024-09-17T23:31:42.942997+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:31:42.942997+0000","last_undegraded":"2024-09-17T23:31:42.942997+0000","last_fullsized":"2024-09-17T23:31:42.942997+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:42.209865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051177199999999996,"stat_sum":{"num_bytes":87677952,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":5018,"num_read_kb":38008,"num_write":477,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"b"}]},{"pgid":"3.13","version":"132'543","reported_seq":15010,"reported_epoch":138,"state":"peering","last_fresh":"2024-09-17T23:31:42.048994+0000","last_change":"2024-09-17T23:31:42.943115+0000","last_active":"2024-09-17T23:31:42.943115+0000","last_peered":"2024-09-17T23:31:42.048994+0000","last_clean":"2024-09-17T23:31:37.865398+0000","last_became_active":"2024-09-17T23:31:40.903890+0000","last_became_peered":"2024-09-17T23:31:40.903890+0000","last_unstale":"2024-09-17T23:31:42.048994+0000","last_undegraded":"2024-09-17T23:31:42.048994+0000","last_fullsized":"2024-09-17T23:31:40.868169+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.010021891999999999,"stat_sum":{"num_bytes":49503337,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":15068,"num_read_kb":74824,"num_write":702,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"22","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1078111429,"num_objects":1328,"num_object_clones":343,"num_object_copies":2656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1328,"num_whiteouts":560,"num_read":306873,"num_read_kb":1321458,"num_write":18664,"num_write_kb":480589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6027,"num_bytes_recovered":240648719,"num_keys_recovered":1110,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32821248,"data_stored":224114191,"data_compressed":11942570,"data_compressed_allocated":24494080,"data_compressed_original":219471872,"omap_allocated":7598,"internal_metadata":0},"log_size":16874,"ondisk_log_size":16874,"up":63,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500679,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43672,"kb_used_data":6968,"kb_used_omap":19,"kb_used_meta":36652,"kb_avail":94328168,"statfs":{"total":96636764160,"available":96592044032,"internally_reserved":0,"allocated":7135232,"data_stored":37201822,"data_compressed":1903848,"data_compressed_allocated":4071424,"data_compressed_original":35709440,"omap_allocated":19797,"internal_metadata":37532331},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640292,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66524,"kb_used_data":8956,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":9170944,"data_stored":52756286,"data_compressed":2864136,"data_compressed_allocated":5750784,"data_compressed_original":51123712,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75416,"kb_used_data":8136,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94296424,"statfs":{"total":96636764160,"available":96559538176,"internally_reserved":0,"allocated":8331264,"data_stored":48486976,"data_compressed":2427607,"data_compressed_allocated":4902912,"data_compressed_original":46795776,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640292,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640293,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77944,"kb_used_data":6500,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293896,"statfs":{"total":96636764160,"available":96556949504,"internally_reserved":0,"allocated":6656000,"data_stored":35889756,"data_compressed":1725958,"data_compressed_allocated":3715072,"data_compressed_original":34462208,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5005312,"data_stored":35201006,"data_compressed":1725958,"data_compressed_allocated":3715072,"data_compressed_original":34462208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2887680,"data_stored":19518108,"data_compressed":1045622,"data_compressed_allocated":2232320,"data_compressed_original":19136512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6684672,"data_stored":47798245,"data_compressed":2427607,"data_compressed_allocated":4902912,"data_compressed_original":46795776,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7471104,"data_stored":51472002,"data_compressed":2858358,"data_compressed_allocated":5713920,"data_compressed_original":50533888,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5468160,"data_stored":35939730,"data_compressed":1898906,"data_compressed_allocated":4034560,"data_compressed_original":35136000,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:31:44.635 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [1, 3, 2, 0] 2024-09-17T23:31:44.635 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1f', '1', '3', '2', '0'] 2024-09-17T23:31:44.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1f 1 3 2 0 2024-09-17T23:31:44.671 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 165.19679182600001, 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:44.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:44.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.338956013, 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:44.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:44.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33826372, 2024-09-17T23:31:44.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:44.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33825213900001, 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.337724892, 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:44.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:44.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.337690909, 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:44.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.337380317, 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:44.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.337354183, 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33685627600001, 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:44.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:44.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.335706551, 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:44.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:44.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.335498953, 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:44.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:44.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.32588305199999, 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:44.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:44.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.32587281799999, 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.325862951, 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:44.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.325852803, 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:44.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.32538293499999, 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.325360819, 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:44.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:44.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:44.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:44.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.139686954, 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.766090327000001, 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:31:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:31:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:31:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e136)", 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8035029790000001, 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031736553, 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23425 2024-09-17T23:31:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864730+0000", 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.485e-06 2024-09-17T23:31:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864735+0000", 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864747+0000", 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1834999999999999e-05 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864758+0000", 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0431000000000001e-05 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864829+0000", 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1526000000000006e-05 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864837+0000", 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2509999999999994e-06 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867486+0000", 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002648376 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867499+0000", 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3553999999999999e-05 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896137+0000", 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.028637427 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896249+0000", 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011187 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896465+0000", 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021632800000000001 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:44.766 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:44.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:44.769 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:44.765+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:44.868 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:44.866+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:44.919 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:44.919 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:44.919 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:44.919 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:44.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:44.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:31:44.972 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:44.969+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:45.048 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:45.049 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:45.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:45.069 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:45.067+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:45.170 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:45.170 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:45.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:45.170 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 125.13156106300001, 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:45.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:45.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 125.13138812299999, 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:45.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:45.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:45.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:45.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 125.131348253, 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:45.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 124.83684402599999, 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:45.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:45.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.589142477, 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.579210211, 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:45.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.56598316199999, 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.025454006, 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.025432616, 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.025415487, 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:45.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:45.221 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.025398205, 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:45.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:45.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:45.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.025381176, 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:45.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:45.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.025348302, 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:45.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.02476059599999, 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:45.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:45.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 98.002968455000001, 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:45.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 98.002431267000006, 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:45.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:45.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 98.001945151000001, 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:45.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.929802549000001, 2024-09-17T23:31:45.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:45.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:45.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.929780738000005, 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:45.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:45.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.920337868000004, 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:45.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:45.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:45.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:45.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:45.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:45.270 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:45.272 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:45.168+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:45.272 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:45.270+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:45.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:45.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:45.373 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:45.371+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:45.383 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:45.383 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:45.383 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:45.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:45.414 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:45.414 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:45.431 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:45.447 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:45.448 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:45.465 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:45.465 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:45.473 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:45.471+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:45.483 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:45.483 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:45.500 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:45.500 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:45.541 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:45.541 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:45.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:45.594 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:45.592+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:45.657 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:45.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:45.707 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:45.705+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:45.777 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:45.774+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:45.813 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:45.813 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:45.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.1f pg_upmap_items mapping to [1->3,2->0] 2024-09-17T23:31:45.858 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:31:45.878 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:45.876+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:45.938 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:45.938 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:45.948 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:45.989 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:45.987+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:46.081 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:46.080+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:46.096 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.096 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:46.182 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:46.180+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:46.253 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.253 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.261 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:46.305 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:46.303+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:46.368 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.369 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:46.419 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:46.417+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:46.485 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:46.482+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:46.525 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.528 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:46.585 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:46.582+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:46.655 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.655 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:46.686 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:46.685+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:46.787 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:46.786+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:46.814 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.814 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:46.889 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:46.887+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:46.931 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:46.931 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:46.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:46.989 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:46.987+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:47.054 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:47.055 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:47.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:47.090 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:47.088+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:47.191 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:47.189+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:47.211 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:47.211 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:47.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:47.292 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:47.291+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:47.368 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:47.368 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:47.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:47.420 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:47.418+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:47.484 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:47.484 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:47.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:47.495 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:47.493+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:47.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:47.595 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:47.594+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:47.645 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:47.646 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:47.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:47.666 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:47.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:47.684 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:47.696 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:47.694+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:47.700 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:47.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:47.718 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:47.736 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:47.736 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:47.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:47.753 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.730057078, 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:47.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:47.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.729884138, 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:47.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:47.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.72984426799999, 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:47.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.435340041, 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:47.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:47.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.187638492, 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:47.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:47.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:47.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 114.177706226, 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:47.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 114.164479177, 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:47.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.806 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.623950021, 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:47.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:47.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:47.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.623928631, 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:47.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.623911502, 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:47.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.62389422, 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:47.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.62387719100001, 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:47.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:47.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:47.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.62384431700001, 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:47.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:47.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.623256611, 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:47.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.60146447, 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:47.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:47.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.600927282, 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:47.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:47.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.600441166, 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:47.854 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.528298563999996, 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:47.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:47.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:47.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.528276753, 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:47.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.518833882999999, 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:47.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:47.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:47.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:47.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:47.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:47.876 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:47.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:31:47.878 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:47.876+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:47.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:47.978 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:47.976+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:48.005 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:48.006 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:48.006 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:48.006 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:48.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:48.100 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:48.098+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:48.169 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:48.169 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:48.170 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:48.170 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:48.170 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:48.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:48.179 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:48.176+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:48.280 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:48.278+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:48.327 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:48.327 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:48.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:48.327 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:48.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.449047729, 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.328 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:48.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.332 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.333 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.43342259900001, 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:48.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:48.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:48.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.43338494299999, 2024-09-17T23:31:48.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:48.340 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.340 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.340 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.344 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.345 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.43336751199999, 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:48.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.350 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.433332581, 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:48.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:48.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.433301161, 2024-09-17T23:31:48.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:48.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:48.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.43328453500001, 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.362 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:48.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:48.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.43326284099999, 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.367 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 141.43325112299999, 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:48.372 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:48.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:48.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:48.377 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.474112603999998, 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:48.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:48.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:48.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.384 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.470157439999994, 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:48.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:48.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:48.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:48.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.470116782000005, 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.390 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.392 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.469857356999995, 2024-09-17T23:31:48.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:48.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:48.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.469821135000004, 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:48.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:48.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:48.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.469788840000007, 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.406 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:48.407 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.408 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:48.409 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.410 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.411 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.469634029000005, 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.412 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.413 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:48.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.415 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.416 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 82.469347709000004, 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 82.469279048000004, 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 82.469237074000006, 2024-09-17T23:31:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.432 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.469453050999999, 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.433 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.434 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:48.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:48.436 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.437 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.438 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:48.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:48.440 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:48.438+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:48.541 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:48.538+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:48.590 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:48.591 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:48.591 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:48.591 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:48.599 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:48.642 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:48.639+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:48.706 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:48.706 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:48.706 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:48.706 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:48.715 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:48.743 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:48.740+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.326875941, 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:48.824 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:48.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.326856061, 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.833 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.834 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.326834523, 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:48.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.839 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.32494180899999, 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:48.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.32492957900001, 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:48.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:48.847 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.848 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.32491901, 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.850 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.851 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:48.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.32490912200001, 2024-09-17T23:31:48.853 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:48.854 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.855 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:48.857 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.32489884, 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.858 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.860 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.861 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:48.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.32488295300001, 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:48.865 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.866 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.867 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:48.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.869 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.869 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.869 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:48.869 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.78595763600001, 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:48.871 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.872 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:48.873 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.874 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.874 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.874 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:48.874 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:48.876 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.877 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.878 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.78388825, 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:48.879 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.881 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.882 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.883 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:48.884 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.452020882, 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:48.885 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:48.886 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.887 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.888 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:48.889 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.967056204000002, 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.890 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.891 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:48.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:48.894 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.967023834000003, 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.895 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:48.896 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.897 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:48.899 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.966731456999995, 2024-09-17T23:31:48.900 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:48.902 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.903 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.960907319, 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:48.907 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:48.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.909 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.960865815999995, 2024-09-17T23:31:48.911 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.912 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:48.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:48.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.916 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.918127549000005, 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.917 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:48.919 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:48.920 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.921 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 87.918025208000003, 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.923 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:48.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:48.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.9377113060000006, 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:48.928 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:48.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:31:48.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.932 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.933 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.933 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:48.933 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:48.933 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:48.934 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:48.935 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:48.931+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:48.979 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:31:49.038 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:49.035+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:49.052 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:49.053 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:49.053 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:49.053 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:49.063 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:49.136 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:49.133+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:49.178 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:49.179 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:49.187 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:49.237 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:49.235+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:49.296 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.296 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1209 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 15597568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 33.088420143999997, 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044407630000000003, 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1209 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204728+0000", 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204729+0000", 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.077e-06 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204736+0000", 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7889999999999997e-06 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204747+0000", 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0793e-05 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204754+0000", 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3379999999999999e-06 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204820+0000", 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6235000000000005e-05 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.204911+0000", 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1024000000000001e-05 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.205182+0000", 2024-09-17T23:31:49.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000270134 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.208700+0000", 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0035188810000000002 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209111+0000", 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00041066099999999999 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209144+0000", 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2653000000000001e-05 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.209169+0000", 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5177999999999998e-05 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.302 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.913080931000003, 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:49.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:49.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5594.0:1643 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [stat,truncate 14811136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.905446791999999, 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0042815140000000002, 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.307 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5594", 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1207638553", 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 1643 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387701+0000", 2024-09-17T23:31:49.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387703+0000", 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.308e-06 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387708+0000", 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.865e-06 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387723+0000", 2024-09-17T23:31:49.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.556e-05 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387732+0000", 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5099999999999998e-06 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387795+0000", 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2929999999999995e-05 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.387897+0000", 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000102815 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388033+0000", 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013546899999999999 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.388921+0000", 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00088850099999999998 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391906+0000", 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002984202 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391944+0000", 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8565000000000001e-05 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.391983+0000", 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8788999999999997e-05 2024-09-17T23:31:49.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21422 3.0 3:02e09248:::rbd_data.1152d7c8a97e.0000000000000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.312398475000002, 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044109429999999996, 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.313 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21422 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:49.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980750+0000", 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980751+0000", 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.315 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980755+0000", 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2690000000000001e-06 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980768+0000", 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2862e-05 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980776+0000", 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8359999999999992e-06 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980833+0000", 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6749000000000002e-05 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980879+0000", 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6050999999999997e-05 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.980977+0000", 2024-09-17T23:31:49.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8488999999999998e-05 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982120+0000", 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001143257 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985096+0000", 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00297569 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985133+0000", 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7134000000000001e-05 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.985161+0000", 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7514000000000001e-05 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:49.319 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.310829879000003, 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:49.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:49.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 31.210197242, 2024-09-17T23:31:49.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:49.327 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.328 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.329 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:49.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:49.331 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22086 3.0 3:04a20ea7:::rbd_data.1152c10d4b9c.000000000000000b:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.062210671999999, 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0043112539999999996, 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.332 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22086 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.333 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230938+0000", 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230939+0000", 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3370000000000001e-06 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230945+0000", 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.37e-06 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.334 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230966+0000", 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0492999999999998e-05 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.230991+0000", 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5463e-05 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232196+0000", 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012043069999999999 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.335 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232366+0000", 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00017094000000000001 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.232576+0000", 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020904100000000001 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.233937+0000", 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0013611140000000001 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.336 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235062+0000", 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001125175 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235117+0000", 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4976999999999999e-05 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235249+0000", 2024-09-17T23:31:49.337 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000132037 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.057669596, 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.338 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.340 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.341 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.342 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:31:49.343 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.057532356999999, 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.344 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:49.345 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.346 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.347 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.348 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:31:49.349 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.052375197, 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.350 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:31:49.352 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.353 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.354 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.355 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22907 3.0 3:07f1f9a0:::rbd_data.11524e94aa61.0000000000000007:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.317321212, 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053158520000000003, 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22907 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.357 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975829+0000", 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975838+0000", 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3260000000000002e-06 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975859+0000", 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0378999999999999e-05 2024-09-17T23:31:49.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975870+0000", 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1488000000000001e-05 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975948+0000", 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199000000000001e-05 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976110+0000", 2024-09-17T23:31:49.359 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000161977 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976234+0000", 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012316599999999999 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.977277+0000", 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010435679999999999 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.360 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981028+0000", 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037512130000000002 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981092+0000", 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3468999999999998e-05 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981143+0000", 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1017999999999998e-05 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.361 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22969 3.17 3:e879c454:::rbd_header.1152324fa224:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.404151770999999, 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058282270000000001, 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22969 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.363 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888998+0000", 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889005+0000", 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000004e-06 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889017+0000", 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2478e-05 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.364 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889029+0000", 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.168e-05 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889311+0000", 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028192400000000002 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889435+0000", 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000124046 2024-09-17T23:31:49.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889886+0000", 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00045141000000000002 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.892147+0000", 2024-09-17T23:31:49.366 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002260813 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894710+0000", 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025627710000000001 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894768+0000", 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8059000000000001e-05 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.367 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894825+0000", 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6986e-05 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23066 3.1f 3:fea969d6:::rbd_data.1152324fa224.0000000000000007:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.320515753999999, 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048693310000000002, 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23066 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.369 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.569999999999999e-07 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972638+0000", 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8040000000000001e-06 2024-09-17T23:31:49.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972649+0000", 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0882e-05 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972656+0000", 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973702+0000", 2024-09-17T23:31:49.371 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010456809999999999 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973830+0000", 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128071 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973960+0000", 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013065799999999999 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:49.372 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.975260+0000", 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012997729999999999 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977401+0000", 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021405030000000002 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977457+0000", 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168999999999997e-05 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.373 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977502+0000", 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4919000000000001e-05 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23135 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'391, mlcod=0'0)", 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.189939904999999, 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053043279999999996, 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.374 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103209+0000", 2024-09-17T23:31:49.375 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103215+0000", 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999998e-06 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103222+0000", 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4919999999999996e-06 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.376 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103231+0000", 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6049999999999998e-06 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103277+0000", 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6641999999999998e-05 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103302+0000", 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4601e-05 2024-09-17T23:31:49.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108442+0000", 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051404129999999999 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108513+0000", 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0192999999999997e-05 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23195 3.1c e131/130 3:3831950c:::rbd_directory:head v 131'414, mlcod=0'0)", 2024-09-17T23:31:49.378 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.108034204999999, 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047496070000000003, 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.379 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185114+0000", 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185115+0000", 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1900000000000003e-07 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185119+0000", 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9230000000000001e-06 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.380 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185128+0000", 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0530000000000007e-06 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185132+0000", 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3889999999999998e-06 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185180+0000", 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8059000000000002e-05 2024-09-17T23:31:49.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.185205+0000", 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4865999999999999e-05 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.189830+0000", 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0046256379999999996 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.189864+0000", 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3160000000000001e-05 2024-09-17T23:31:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23232 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'398, mlcod=23'1)", 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.061403015, 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048635629999999996, 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.383 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231746+0000", 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999996e-07 2024-09-17T23:31:49.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231752+0000", 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7790000000000001e-06 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231760+0000", 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6279999999999994e-06 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.385 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231765+0000", 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8919999999999999e-06 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231809+0000", 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3544000000000002e-05 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231837+0000", 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7815999999999999e-05 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.386 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236575+0000", 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047384030000000004 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236609+0000", 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3896999999999997e-05 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.387 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23275 3.1 e132/130 3:86cbfba3:::rbd_header.11528a3e36a5:head v 132'803, mlcod=23'42)", 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.861153+0000", 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 13.431995269, 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0044164490000000002, 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861153+0000", 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861153+0000", 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861154+0000", 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1600000000000001e-07 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861160+0000", 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3790000000000003e-06 2024-09-17T23:31:49.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861168+0000", 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2460000000000003e-06 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861173+0000", 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.95e-06 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861225+0000", 2024-09-17T23:31:49.390 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1730000000000001e-05 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.861249+0000", 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4307e-05 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.865288+0000", 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0040388239999999999 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.391 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.865569+0000", 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028149700000000002 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23659 3.11 e140/138 3:8d878811:::rbd_data.115258848ebb.0000000000000005:head v 140'344, mlcod=0'0)", 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:44.020817+0000", 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 5.2723308830000004, 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0041208349999999998, 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.020817+0000", 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.020817+0000", 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.393 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.020818+0000", 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1200000000000002e-07 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.020826+0000", 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.020840+0000", 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3930999999999999e-05 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.394 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.020847+0000", 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3610000000000003e-06 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.021001+0000", 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000154091 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.021019+0000", 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.791e-05 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.395 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.024908+0000", 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038886189999999998 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:44.024938+0000", 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.0264e-05 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.396 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24441 3.4 e145/133 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:head v 145'377, mlcod=137'375)", 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:49.074457+0000", 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.21869129900000001, 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0041931470000000004, 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074457+0000", 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.397 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074457+0000", 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074458+0000", 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9599999999999999e-07 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074464+0000", 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4749999999999998e-06 2024-09-17T23:31:49.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074476+0000", 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1644999999999999e-05 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074483+0000", 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4080000000000001e-06 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074539+0000", 2024-09-17T23:31:49.399 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5534000000000001e-05 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.074554+0000", 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4819e-05 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.078602+0000", 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0040488069999999998 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.400 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.078650+0000", 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7763000000000003e-05 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24442 3.11 e145/138 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:head v 145'346, mlcod=140'344)", 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.21749563299999999, 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054889680000000003, 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.401 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:49.402 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7499999999999999e-07 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075660+0000", 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3430000000000001e-06 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075673+0000", 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2758000000000001e-05 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.403 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075680+0000", 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0190000000000001e-06 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075731+0000", 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1872000000000001e-05 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075755+0000", 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3668000000000001e-05 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.404 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081093+0000", 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053383019999999996 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081142+0000", 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8231e-05 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:49.405 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:49.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:49.407 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:49.404+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:49.508 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:49.506+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:49.526 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:49.527 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:49.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:49.608 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:49.606+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:49.644 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:49.644 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:49.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:49.709 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:49.707+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:49.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:49.804 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:49.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.959388232999999, 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:49.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:49.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.949816134000002, 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.811 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.946285627999998, 2024-09-17T23:31:49.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:49.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:49.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.944790921999996, 2024-09-17T23:31:49.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:49.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:49.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.944608002999999, 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:49.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.944494501999998, 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:49.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:49.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.944470394999996, 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.944396064000003, 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:49.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.944372514999998, 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:49.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:49.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.941201313999997, 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:49.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.936792978, 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:49.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:49.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:49.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.909520870999998, 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:49.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:49.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:49.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.900338719000004, 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:49.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:49.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:49.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.900307225999995, 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:49.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:49.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.900281534000001, 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:49.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:49.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.900136982000006, 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:49.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.898568865000001, 2024-09-17T23:31:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:49.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.898371518000005, 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:49.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:49.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.898345203000005, 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:49.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:49.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.898316859999994, 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:49.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:49.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:49.912 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:49.913 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:49.914 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:49.914 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:49.915 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:49.911+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:49.921 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:49.938 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:49.938 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:49.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:49.956 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:49.972 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:49.973 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:50.028 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:50.025+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:50.035 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.035 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:50.130 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:50.127+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:50.195 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.195 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:50.246 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:50.243+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:50.316 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:50.315+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:50.352 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.352 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:50.417 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:50.415+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:50.469 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.469 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:50.519 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:50.517+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:50.620 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:50.618+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.626 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:50.721 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:50.717+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:50.743 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.743 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:50.822 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:50.819+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:50.872 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.872 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.875 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:31:50.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T23:31:50.881 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:50.925 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:50.923+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:50.990 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:50.990 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:50.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:31:51.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:51.044 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:51.042+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:51.124 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:51.122+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:51.160 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:51.160 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:51.169 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:51.253 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:51.251+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:51.316 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:51.317 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:51.325 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:51.366 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:51.364+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:51.427 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:51.425+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:51.472 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:51.472 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:51.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:51.529 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:51.526+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:51.588 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:51.588 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:51.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:51.640 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:51.638+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:51.704 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:51.704 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:51.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:51.754 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:51.752+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:51.830 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:51.829+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:51.860 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:51.860 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:51.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:51.953 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:51.951+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:51.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:51.994 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:31:51.996 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T23:31:52.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T23:31:52.022 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:52.022 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:52.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:52.059 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:52.056+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:52.137 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:52.137 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:52.137 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:52.135+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:52.154 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:52.154 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:52.170 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:52.170 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:52.181 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:52.182 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:52.182 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:52.188 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:52.189 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:52.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:52.205 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:52.206 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:52.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:52.223 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:52.278 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:52.275+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:52.316 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:52.316 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:52.317 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:52.317 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:52.325 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:52.340 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:52.339+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:52.441 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:52.438+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:52.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.629864014000006, 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.620291914999996, 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.616761409000006, 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.615266703000003, 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.615083784000007, 2024-09-17T23:31:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.614970283000005, 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.614946176000004, 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.614871844999996, 2024-09-17T23:31:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.614848296000005, 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.611677095000005, 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.607268758999993, 2024-09-17T23:31:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.579996652000005, 2024-09-17T23:31:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.570814499999997, 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.570783007000003, 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.570757314999994, 2024-09-17T23:31:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.570612763, 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.569044645999995, 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.568847298999998, 2024-09-17T23:31:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:52.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.568820983999998, 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:52.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:52.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.568792641000002, 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.579 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:52.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:52.581 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:52.578+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:52.682 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:52.680+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:52.728 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:52.728 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:52.728 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:52.728 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:52.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:52.783 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:52.781+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:52.843 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:52.844 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:52.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:52.884 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:52.882+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:52.961 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:52.961 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:52.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:52.961 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:52.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 173.48690185500001, 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:31:52.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:31:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.62906604200001, 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.628373749, 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:31:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:31:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.628362168, 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:31:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.627834921, 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:31:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.62780093800001, 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.627490346, 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:31:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.62746421200001, 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:31:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:31:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.626966305, 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:31:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.62581658000001, 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.625608982, 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:31:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:31:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.615993081, 2024-09-17T23:31:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:31:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:31:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.615982847, 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:31:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:31:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.61597298, 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:31:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.61596283199999, 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:31:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.615492964, 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:31:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:31:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.615470848, 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:31:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.429796983, 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:31:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:31:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:31:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:31:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.056200356000002, 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e136)", 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.093613008, 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031736553, 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23425 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:31:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864730+0000", 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.485e-06 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864735+0000", 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864747+0000", 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1834999999999999e-05 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864758+0000", 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0431000000000001e-05 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864829+0000", 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1526000000000006e-05 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864837+0000", 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2509999999999994e-06 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867486+0000", 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002648376 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867499+0000", 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3553999999999999e-05 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896137+0000", 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.028637427 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896249+0000", 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011187 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896465+0000", 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021632800000000001 2024-09-17T23:31:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.057 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:53.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T23:31:53.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T23:31:53.061 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:53.057+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:53.160 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:53.156+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:53.210 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:53.210 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:53.211 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:53.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:53.261 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:53.258+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:53.362 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:53.359+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:53.368 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:53.368 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:53.368 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:53.368 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:53.368 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:53.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:53.463 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:53.462+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.487286372, 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:31:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:31:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:31:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.487113432, 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:31:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:31:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.48707356200001, 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:31:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:31:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.192569335, 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:31:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:31:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:31:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:31:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:31:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.944867786, 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:31:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.93493552, 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:31:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:31:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:31:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:31:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:31:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:31:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:31:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:31:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.921708471, 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:31:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:31:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.381179315, 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:31:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.381157925, 2024-09-17T23:31:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:31:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.381140796, 2024-09-17T23:31:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.381123514, 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.381106485, 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:31:53.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:31:53.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.38107361100001, 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:31:53.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:31:53.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.380485905, 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:31:53.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 106.35869376399999, 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:31:53.608 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:31:53.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 106.358156576, 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:31:53.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 106.35767045999999, 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:31:53.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:31:53.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.285527857999995, 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:31:53.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.285506046999998, 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:31:53.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.276063176999997, 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:53.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:31:53.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:53.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:53.638 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:53.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:53.640 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:53.637+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:53.740 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:53.738+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:53.760 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:24856 3.b 3.3172c0cb (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.75359124700000002, 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.75361380200000005, 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "waiting for peered", 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:53.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 24856 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:31:53.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005662+0000", 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9350000000000001e-06 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005669+0000", 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3470000000000004e-06 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:53.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005683+0000", 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4419999999999999e-05 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005694+0000", 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0917999999999999e-05 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006157+0000", 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00046328800000000001 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006166+0000", 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 1 2024-09-17T23:31:53.765 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:53.769 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:53.841 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:53.838+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:53.879 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:53.879 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:53.879 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:53.879 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:53.879 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:53.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:53.942 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:53.940+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:53.997 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.11980893, 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:53.998 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:53.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:54.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:54.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.003 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.10418379999999, 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:54.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.104146144, 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:54.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:54.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.10412871299999, 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:54.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:54.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:54.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.10409378200001, 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.021 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:54.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:54.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.10406236200001, 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:54.026 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.031 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.10404573599999, 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:54.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.10402404199999, 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.037 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:54.038 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:54.039 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.040 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.041 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.042 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 147.104012324, 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.043 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:54.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:54.046 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:54.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.048 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.144873805000003, 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:54.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.051 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.052 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:54.055 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.140918640999999, 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.056 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.057 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.058 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.060 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:54.061 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.140877982999996, 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:54.063 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.064 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.065 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:523 3.11 3:8f65b468:::rbd_data.1152adcf8b1e.0000000000000008:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:54.066 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.140618558, 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014550769999999, 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 523 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853937+0000", 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-07 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853941+0000", 2024-09-17T23:31:54.068 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.36e-06 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853952+0000", 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.146e-05 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853959+0000", 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8539999999999999e-06 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854297+0000", 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00033791000000000002 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854304+0000", 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1609999999999997e-06 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855209+0000", 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.000904676 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855284+0000", 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5704999999999994e-05 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855392+0000", 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010738699999999999 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.140582335999994, 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:54.073 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.074 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.075 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.076 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.140550040999997, 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:54.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.140395229999996, 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:54.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 88.140108909999995, 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.088 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:54.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:54.092 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 88.140040248999995, 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:54.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.095 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.097 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 88.139998274999996, 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:54.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:54.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 33.140214252, 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:54.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:54.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.108 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:54.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:31:54.111 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T23:31:54.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:54.112 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:54.108+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:54.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T23:31:54.211 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:54.208+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:54.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:54.274 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:54.274 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:54.275 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:54.275 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:54.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:54.312 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:54.309+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:54.405 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:54.406 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:54.412 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:54.412 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:54.412 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:54.412 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:54.412 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:54.413 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:54.410+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:54.421 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:54.441 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:54.442 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:54.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:54.476 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:54.511 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:54.511 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:54.513 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:54.511+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:54.541 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.04440487799999, 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.542 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:54.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.546 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.044384998, 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:54.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:54.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.04436346, 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.551 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:54.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.555 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.04247074599999, 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.556 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:54.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.04245851600001, 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.561 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.562 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:54.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.042447947, 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.565 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:54.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.567 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:54.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.04243805900001, 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:54.569 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:54.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.042427777, 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:54.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:54.578 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.04241189000001, 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:54.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:54.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 116.503486573, 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:54.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:54.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 116.501417187, 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:54.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:54.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:54.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.597 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.169549819, 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.598 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:54.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.602 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.684585140999999, 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:54.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.606 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.607 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.684552771, 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:54.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.684260394000006, 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:54.613 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:54.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:54.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:54.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.678436255999998, 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:54.618 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.619 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:54.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:54.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:54.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:54.623 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.678394753000006, 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.635656486000002, 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:54.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.630 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:54.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:54.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 93.635554145, 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.634 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:54.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 15.655240243, 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:54.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:54.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:54.641 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.643 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:54.644 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:54.644 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:54.645 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:54.645 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:54.646 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:54.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:54.647 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:54.643+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:54.746 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:54.744+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:54.798 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:54.798 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:54.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:54.846 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:54.844+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:54.947 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:54.945+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:54.964 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:54.964 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:54.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:55.001 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:31:55.007 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T23:31:55.025 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T23:31:55.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:31:55.067 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:55.064+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:55.103 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:55.103 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:55.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:55.157 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:55.155+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:55.235 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:55.235 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:55.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:55.249 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:55.248+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:55.350 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:55.348+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:55.409 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:55.409 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:55.412 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:55.413 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:31:55.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:55.426 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-17T23:31:54.998526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":798035561,"num_objects":929,"num_object_clones":202,"num_object_copies":1858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":929,"num_whiteouts":407,"num_read":284591,"num_read_kb":1250698,"num_write":17772,"num_write_kb":468648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5297,"num_bytes_recovered":225317271,"num_keys_recovered":888,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16113,"ondisk_log_size":16113,"up":81,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":272224,"kb_used_data":26720,"kb_used_omap":24,"kb_used_meta":245351,"kb_avail":377215136,"statfs":{"total":386547056640,"available":386268299264,"internally_reserved":0,"allocated":27361280,"data_stored":148935031,"data_compressed":7278247,"data_compressed_allocated":14880768,"data_compressed_original":142962176,"omap_allocated":24578,"internal_metadata":251240446},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-138866866,"num_objects":-349,"num_object_clones":-134,"num_object_copies":-698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-349,"num_whiteouts":-146,"num_read":-23612,"num_read_kb":-71899,"num_write":-1260,"num_write_kb":-14423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-742,"num_bytes_recovered":-15921892,"num_keys_recovered":-222,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010629"},"pg_stats":[{"pgid":"3.1f","version":"146'413","reported_seq":5731,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:50.994093+0000","last_change":"2024-09-17T23:31:53.003682+0000","last_active":"2024-09-17T23:31:53.003682+0000","last_peered":"2024-09-17T23:31:50.994093+0000","last_clean":"2024-09-17T23:31:50.994093+0000","last_became_active":"2024-09-17T23:31:43.946390+0000","last_became_peered":"2024-09-17T23:31:43.946390+0000","last_unstale":"2024-09-17T23:31:50.994093+0000","last_undegraded":"2024-09-17T23:31:50.994093+0000","last_fullsized":"2024-09-17T23:31:50.994093+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001386211,"stat_sum":{"num_bytes":31087668,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":7434,"num_read_kb":66454,"num_write":453,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":21604086,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,5],"up_primary":3,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1e","version":"146'525","reported_seq":8881,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.999169+0000","last_change":"2024-09-17T23:31:51.996746+0000","last_active":"2024-09-17T23:31:52.999169+0000","last_peered":"2024-09-17T23:31:52.999169+0000","last_clean":"2024-09-17T23:31:52.999169+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:31:52.999169+0000","last_undegraded":"2024-09-17T23:31:52.999169+0000","last_fullsized":"2024-09-17T23:31:52.999169+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:29.234633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048813699999999998,"stat_sum":{"num_bytes":80398045,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":9314,"num_read_kb":55080,"num_write":535,"num_write_kb":19041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.1d","version":"146'345","reported_seq":5781,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001765+0000","last_change":"2024-09-17T23:31:52.000880+0000","last_active":"2024-09-17T23:31:53.001765+0000","last_peered":"2024-09-17T23:31:53.001765+0000","last_clean":"2024-09-17T23:31:53.001765+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:31:53.001765+0000","last_undegraded":"2024-09-17T23:31:53.001765+0000","last_fullsized":"2024-09-17T23:31:53.001765+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:37.132959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001352109,"stat_sum":{"num_bytes":16818688,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4805,"num_read_kb":41204,"num_write":320,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.1c","version":"147'428","reported_seq":9083,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.111570+0000","last_change":"2024-09-17T23:31:53.004165+0000","last_active":"2024-09-17T23:31:53.004165+0000","last_peered":"2024-09-17T23:31:51.111570+0000","last_clean":"2024-09-17T23:31:51.111570+0000","last_became_active":"2024-09-17T23:31:42.946855+0000","last_became_peered":"2024-09-17T23:31:42.946855+0000","last_unstale":"2024-09-17T23:31:51.111570+0000","last_undegraded":"2024-09-17T23:31:51.111570+0000","last_fullsized":"2024-09-17T23:31:51.111570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403729,"stat_sum":{"num_bytes":14477328,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":8439,"num_read_kb":40187,"num_write":516,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":14561502,"num_keys_recovered":350,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1b","version":"147'413","reported_seq":19288,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995904+0000","last_change":"2024-09-17T23:31:52.999783+0000","last_active":"2024-09-17T23:31:51.995904+0000","last_peered":"2024-09-17T23:31:51.138296+0000","last_clean":"2024-09-17T23:31:51.138296+0000","last_became_active":"2024-09-17T23:31:37.864076+0000","last_became_peered":"2024-09-17T23:31:37.864076+0000","last_unstale":"2024-09-17T23:31:51.995904+0000","last_undegraded":"2024-09-17T23:31:51.995904+0000","last_fullsized":"2024-09-17T23:31:51.995904+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062600000000000004,"stat_sum":{"num_bytes":62681915,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":18867,"num_read_kb":58412,"num_write":454,"num_write_kb":23019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":2836795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1a","version":"146'856","reported_seq":7400,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004831+0000","last_change":"2024-09-17T23:31:52.000256+0000","last_active":"2024-09-17T23:31:53.004831+0000","last_peered":"2024-09-17T23:31:53.004831+0000","last_clean":"2024-09-17T23:31:53.004831+0000","last_became_active":"2024-09-17T23:31:43.945744+0000","last_became_peered":"2024-09-17T23:31:43.945744+0000","last_unstale":"2024-09-17T23:31:53.004831+0000","last_undegraded":"2024-09-17T23:31:53.004831+0000","last_fullsized":"2024-09-17T23:31:53.004831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:54.415718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000452776,"stat_sum":{"num_bytes":31380669,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":11922,"num_read_kb":30828,"num_write":1189,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.19","version":"146'803","reported_seq":6496,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995822+0000","last_change":"2024-09-17T23:31:52.999112+0000","last_active":"2024-09-17T23:31:51.995822+0000","last_peered":"2024-09-17T23:31:50.998697+0000","last_clean":"2024-09-17T23:31:50.998697+0000","last_became_active":"2024-09-17T23:31:42.945986+0000","last_became_peered":"2024-09-17T23:31:42.945986+0000","last_unstale":"2024-09-17T23:31:51.995822+0000","last_undegraded":"2024-09-17T23:31:51.995822+0000","last_fullsized":"2024-09-17T23:31:51.995822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088431689999999997,"stat_sum":{"num_bytes":15988255,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":8532,"num_read_kb":17539,"num_write":911,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":4472153,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.18","version":"146'435","reported_seq":4255,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.997643+0000","last_change":"2024-09-17T23:31:51.995873+0000","last_active":"2024-09-17T23:31:52.997643+0000","last_peered":"2024-09-17T23:31:52.997643+0000","last_clean":"2024-09-17T23:31:52.997643+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:31:52.997643+0000","last_undegraded":"2024-09-17T23:31:52.997643+0000","last_fullsized":"2024-09-17T23:31:52.997643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:59.943227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841602,"stat_sum":{"num_bytes":17261108,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":4986,"num_read_kb":22191,"num_write":433,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.17","version":"146'597","reported_seq":7899,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-17T23:31:50.026446+0000","last_change":"2024-09-17T23:31:50.028992+0000","last_active":"2024-09-17T23:31:50.026446+0000","last_peered":"2024-09-17T23:31:50.026446+0000","last_clean":"2024-09-17T23:31:50.026446+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:31:50.026446+0000","last_undegraded":"2024-09-17T23:31:50.026446+0000","last_fullsized":"2024-09-17T23:31:50.026446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:58:45.677907+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.049582080000000001,"stat_sum":{"num_bytes":36516954,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":10694,"num_read_kb":28755,"num_write":514,"num_write_kb":13263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.16","version":"146'351","reported_seq":4259,"reported_epoch":146,"state":"active+clean+remapped","last_fresh":"2024-09-17T23:31:49.989903+0000","last_change":"2024-09-17T23:31:49.991057+0000","last_active":"2024-09-17T23:31:49.989903+0000","last_peered":"2024-09-17T23:31:49.989903+0000","last_clean":"2024-09-17T23:31:49.989903+0000","last_became_active":"2024-09-17T23:31:40.879305+0000","last_became_peered":"2024-09-17T23:31:40.879305+0000","last_unstale":"2024-09-17T23:31:49.989903+0000","last_undegraded":"2024-09-17T23:31:49.989903+0000","last_fullsized":"2024-09-17T23:31:49.989903+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:50.049462+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.010952870999999999,"stat_sum":{"num_bytes":14573104,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4431,"num_read_kb":31183,"num_write":358,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5911184,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,5","objects":27}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.15","version":"146'581","reported_seq":12749,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001840+0000","last_change":"2024-09-17T23:31:52.002608+0000","last_active":"2024-09-17T23:31:53.001840+0000","last_peered":"2024-09-17T23:31:53.001840+0000","last_clean":"2024-09-17T23:31:53.001840+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:31:53.001840+0000","last_undegraded":"2024-09-17T23:31:53.001840+0000","last_fullsized":"2024-09-17T23:31:53.001840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:40.342815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002366412,"stat_sum":{"num_bytes":27632680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":14519,"num_read_kb":31572,"num_write":615,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.14","version":"147'536","reported_seq":7847,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001410+0000","last_change":"2024-09-17T23:31:52.000543+0000","last_active":"2024-09-17T23:31:53.001410+0000","last_peered":"2024-09-17T23:31:53.001410+0000","last_clean":"2024-09-17T23:31:53.001410+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:31:53.001410+0000","last_undegraded":"2024-09-17T23:31:53.001410+0000","last_fullsized":"2024-09-17T23:31:53.001410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:56:23.071341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001202137,"stat_sum":{"num_bytes":13340257,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":8401,"num_read_kb":33555,"num_write":450,"num_write_kb":18362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001697+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:31:53.001697+0000","last_peered":"2024-09-17T23:31:53.001697+0000","last_clean":"2024-09-17T23:31:53.001697+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:31:53.001697+0000","last_undegraded":"2024-09-17T23:31:53.001697+0000","last_fullsized":"2024-09-17T23:31:53.001697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"147'602","reported_seq":5017,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003390+0000","last_change":"2024-09-17T23:31:51.999436+0000","last_active":"2024-09-17T23:31:53.003390+0000","last_peered":"2024-09-17T23:31:53.003390+0000","last_clean":"2024-09-17T23:31:53.003390+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:31:53.003390+0000","last_undegraded":"2024-09-17T23:31:53.003390+0000","last_fullsized":"2024-09-17T23:31:53.003390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:43.562801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052716000000000004,"stat_sum":{"num_bytes":13084247,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5974,"num_read_kb":22618,"num_write":871,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":282,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003524+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:31:53.003524+0000","last_peered":"2024-09-17T23:31:53.003524+0000","last_clean":"2024-09-17T23:31:53.003524+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:31:53.003524+0000","last_undegraded":"2024-09-17T23:31:53.003524+0000","last_fullsized":"2024-09-17T23:31:53.003524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"147'414","reported_seq":12448,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.005411+0000","last_change":"2024-09-17T23:31:52.002346+0000","last_active":"2024-09-17T23:31:53.005411+0000","last_peered":"2024-09-17T23:31:53.005411+0000","last_clean":"2024-09-17T23:31:53.005411+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:31:53.005411+0000","last_undegraded":"2024-09-17T23:31:53.005411+0000","last_fullsized":"2024-09-17T23:31:53.005411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:38.208779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056097299999999998,"stat_sum":{"num_bytes":15504427,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":13024,"num_read_kb":60475,"num_write":417,"num_write_kb":30569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003656+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:31:53.003656+0000","last_peered":"2024-09-17T23:31:53.003656+0000","last_clean":"2024-09-17T23:31:53.003656+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:31:53.003656+0000","last_undegraded":"2024-09-17T23:31:53.003656+0000","last_fullsized":"2024-09-17T23:31:53.003656+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"146'381","reported_seq":7206,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004920+0000","last_change":"2024-09-17T23:31:52.001130+0000","last_active":"2024-09-17T23:31:53.004920+0000","last_peered":"2024-09-17T23:31:53.004920+0000","last_clean":"2024-09-17T23:31:53.004920+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:31:53.004920+0000","last_undegraded":"2024-09-17T23:31:53.004920+0000","last_fullsized":"2024-09-17T23:31:53.004920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:01.665826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078735100000000004,"stat_sum":{"num_bytes":14529566,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":4593,"num_read_kb":19709,"num_write":432,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":2,"reported_epoch":149,"state":"peering","last_fresh":"2024-09-17T23:31:52.000114+0000","last_change":"2024-09-17T23:31:53.398866+0000","last_active":"2024-09-17T23:31:53.398866+0000","last_peered":"2024-09-17T23:31:52.000114+0000","last_clean":"2024-09-17T23:31:52.000114+0000","last_became_active":"2024-09-17T23:31:33.918555+0000","last_became_peered":"2024-09-17T23:31:33.918555+0000","last_unstale":"2024-09-17T23:31:52.000114+0000","last_undegraded":"2024-09-17T23:31:52.000114+0000","last_fullsized":"2024-09-17T23:31:52.000114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"147'370","reported_seq":10454,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.998667+0000","last_change":"2024-09-17T23:31:51.996713+0000","last_active":"2024-09-17T23:31:52.998667+0000","last_peered":"2024-09-17T23:31:52.998667+0000","last_clean":"2024-09-17T23:31:52.998667+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:31:52.998667+0000","last_undegraded":"2024-09-17T23:31:52.998667+0000","last_fullsized":"2024-09-17T23:31:52.998667+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:59.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012702,"stat_sum":{"num_bytes":20594208,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":10196,"num_read_kb":27182,"num_write":428,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.3","version":"147'559","reported_seq":7277,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001255+0000","last_change":"2024-09-17T23:31:52.001784+0000","last_active":"2024-09-17T23:31:53.001255+0000","last_peered":"2024-09-17T23:31:53.001255+0000","last_clean":"2024-09-17T23:31:53.001255+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:31:53.001255+0000","last_undegraded":"2024-09-17T23:31:53.001255+0000","last_fullsized":"2024-09-17T23:31:53.001255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:24.363207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020241600000000001,"stat_sum":{"num_bytes":19790336,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":7259,"num_read_kb":29719,"num_write":580,"num_write_kb":9147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.2","version":"18'2","reported_seq":327,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000975+0000","last_change":"2024-09-17T23:31:39.878887+0000","last_active":"2024-09-17T23:31:53.000975+0000","last_peered":"2024-09-17T23:31:53.000975+0000","last_clean":"2024-09-17T23:31:53.000975+0000","last_became_active":"2024-09-17T23:31:39.873374+0000","last_became_peered":"2024-09-17T23:31:39.873374+0000","last_unstale":"2024-09-17T23:31:53.000975+0000","last_undegraded":"2024-09-17T23:31:53.000975+0000","last_fullsized":"2024-09-17T23:31:53.000975+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:49.977116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"unknown","last_fresh":"2024-09-17T23:31:52.999124+0000","last_change":"2024-09-17T23:31:53.398793+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:31:52.999124+0000","last_undegraded":"2024-09-17T23:31:52.999124+0000","last_fullsized":"2024-09-17T23:31:52.999124+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":287,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.998563+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:31:52.998563+0000","last_peered":"2024-09-17T23:31:52.998563+0000","last_clean":"2024-09-17T23:31:52.998563+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:31:52.998563+0000","last_undegraded":"2024-09-17T23:31:52.998563+0000","last_fullsized":"2024-09-17T23:31:52.998563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"unknown","last_fresh":"2024-09-17T23:31:53.000225+0000","last_change":"2024-09-17T23:31:53.398831+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:31:53.000225+0000","last_undegraded":"2024-09-17T23:31:53.000225+0000","last_fullsized":"2024-09-17T23:31:53.000225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002579+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:31:53.002579+0000","last_peered":"2024-09-17T23:31:53.002579+0000","last_clean":"2024-09-17T23:31:53.002579+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:31:53.002579+0000","last_undegraded":"2024-09-17T23:31:53.002579+0000","last_fullsized":"2024-09-17T23:31:53.002579+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"147'670","reported_seq":10958,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995561+0000","last_change":"2024-09-17T23:31:52.999594+0000","last_active":"2024-09-17T23:31:51.995561+0000","last_peered":"2024-09-17T23:31:50.999183+0000","last_clean":"2024-09-17T23:31:50.999183+0000","last_became_active":"2024-09-17T23:31:43.946105+0000","last_became_peered":"2024-09-17T23:31:43.946105+0000","last_unstale":"2024-09-17T23:31:51.995561+0000","last_undegraded":"2024-09-17T23:31:51.995561+0000","last_fullsized":"2024-09-17T23:31:51.995561+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0095413040000000005,"stat_sum":{"num_bytes":23587937,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11181,"num_read_kb":79280,"num_write":755,"num_write_kb":59941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15860611,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"2.3","version":"16'1","reported_seq":205,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004921+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:31:53.004921+0000","last_peered":"2024-09-17T23:31:53.004921+0000","last_clean":"2024-09-17T23:31:53.004921+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:31:53.004921+0000","last_undegraded":"2024-09-17T23:31:53.004921+0000","last_fullsized":"2024-09-17T23:31:53.004921+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002710+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:31:53.002710+0000","last_peered":"2024-09-17T23:31:53.002710+0000","last_clean":"2024-09-17T23:31:53.002710+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:31:53.002710+0000","last_undegraded":"2024-09-17T23:31:53.002710+0000","last_fullsized":"2024-09-17T23:31:53.002710+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"146'430","reported_seq":14649,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002926+0000","last_change":"2024-09-17T23:31:51.999823+0000","last_active":"2024-09-17T23:31:53.002926+0000","last_peered":"2024-09-17T23:31:53.002926+0000","last_clean":"2024-09-17T23:31:53.002926+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:31:53.002926+0000","last_undegraded":"2024-09-17T23:31:53.002926+0000","last_fullsized":"2024-09-17T23:31:53.002926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:12.145593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062045500000000001,"stat_sum":{"num_bytes":30823677,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":15144,"num_read_kb":73922,"num_write":381,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.9","version":"146'695","reported_seq":11364,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.104415+0000","last_change":"2024-09-17T23:31:52.998233+0000","last_active":"2024-09-17T23:31:52.998233+0000","last_peered":"2024-09-17T23:31:51.104415+0000","last_clean":"2024-09-17T23:31:51.104415+0000","last_became_active":"2024-09-17T23:31:36.863448+0000","last_became_peered":"2024-09-17T23:31:36.863448+0000","last_unstale":"2024-09-17T23:31:51.104415+0000","last_undegraded":"2024-09-17T23:31:51.104415+0000","last_fullsized":"2024-09-17T23:31:51.104415+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045313389999999997,"stat_sum":{"num_bytes":12758069,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":11139,"num_read_kb":39046,"num_write":811,"num_write_kb":25901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":1842284,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.a","version":"147'572","reported_seq":9639,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000786+0000","last_change":"2024-09-17T23:31:52.001891+0000","last_active":"2024-09-17T23:31:53.000786+0000","last_peered":"2024-09-17T23:31:53.000786+0000","last_clean":"2024-09-17T23:31:53.000786+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:31:53.000786+0000","last_undegraded":"2024-09-17T23:31:53.000786+0000","last_fullsized":"2024-09-17T23:31:53.000786+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:18.311799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023115129999999998,"stat_sum":{"num_bytes":12512903,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":10779,"num_read_kb":53142,"num_write":576,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.b","version":"147'892","reported_seq":11165,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.996707+0000","last_change":"2024-09-17T23:31:52.999472+0000","last_active":"2024-09-17T23:31:51.996707+0000","last_peered":"2024-09-17T23:31:51.136571+0000","last_clean":"2024-09-17T23:31:51.136571+0000","last_became_active":"2024-09-17T23:31:37.863905+0000","last_became_peered":"2024-09-17T23:31:37.863905+0000","last_unstale":"2024-09-17T23:31:51.996707+0000","last_undegraded":"2024-09-17T23:31:51.996707+0000","last_fullsized":"2024-09-17T23:31:51.996707+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003279213,"stat_sum":{"num_bytes":28522528,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":14913,"num_read_kb":61384,"num_write":876,"num_write_kb":11835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2676806,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.c","version":"147'457","reported_seq":5312,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001521+0000","last_change":"2024-09-17T23:31:52.001483+0000","last_active":"2024-09-17T23:31:53.001521+0000","last_peered":"2024-09-17T23:31:53.001521+0000","last_clean":"2024-09-17T23:31:53.001521+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:31:53.001521+0000","last_undegraded":"2024-09-17T23:31:53.001521+0000","last_fullsized":"2024-09-17T23:31:53.001521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:36.817924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017303850000000001,"stat_sum":{"num_bytes":9347775,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":5958,"num_read_kb":36790,"num_write":450,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.d","version":"147'407","reported_seq":9641,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001837+0000","last_change":"2024-09-17T23:31:52.002602+0000","last_active":"2024-09-17T23:31:53.001837+0000","last_peered":"2024-09-17T23:31:53.001837+0000","last_clean":"2024-09-17T23:31:53.001837+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:31:53.001837+0000","last_undegraded":"2024-09-17T23:31:53.001837+0000","last_fullsized":"2024-09-17T23:31:53.001837+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:23.128610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024799679999999999,"stat_sum":{"num_bytes":26479684,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":9068,"num_read_kb":42813,"num_write":454,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.e","version":"147'695","reported_seq":10003,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.005169+0000","last_change":"2024-09-17T23:31:52.999913+0000","last_active":"2024-09-17T23:31:52.999913+0000","last_peered":"2024-09-17T23:31:51.005169+0000","last_clean":"2024-09-17T23:31:51.005169+0000","last_became_active":"2024-09-17T23:31:38.869556+0000","last_became_peered":"2024-09-17T23:31:38.869556+0000","last_unstale":"2024-09-17T23:31:51.005169+0000","last_undegraded":"2024-09-17T23:31:51.005169+0000","last_fullsized":"2024-09-17T23:31:51.005169+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01512409,"stat_sum":{"num_bytes":17051340,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":10323,"num_read_kb":26354,"num_write":754,"num_write_kb":7190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":3804584,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.f","version":"146'357","reported_seq":4718,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:50.994722+0000","last_change":"2024-09-17T23:31:53.002972+0000","last_active":"2024-09-17T23:31:53.002972+0000","last_peered":"2024-09-17T23:31:50.994722+0000","last_clean":"2024-09-17T23:31:50.994722+0000","last_became_active":"2024-09-17T23:31:37.868909+0000","last_became_peered":"2024-09-17T23:31:37.868909+0000","last_unstale":"2024-09-17T23:31:50.994722+0000","last_undegraded":"2024-09-17T23:31:50.994722+0000","last_fullsized":"2024-09-17T23:31:50.994722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00159764,"stat_sum":{"num_bytes":6860340,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":4817,"num_read_kb":27404,"num_write":372,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":4115627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.10","version":"147'886","reported_seq":7769,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.063703+0000","last_change":"2024-09-17T23:31:52.998040+0000","last_active":"2024-09-17T23:31:52.998040+0000","last_peered":"2024-09-17T23:31:51.063703+0000","last_clean":"2024-09-17T23:31:51.063703+0000","last_became_active":"2024-09-17T23:31:43.944949+0000","last_became_peered":"2024-09-17T23:31:43.944949+0000","last_unstale":"2024-09-17T23:31:51.063703+0000","last_undegraded":"2024-09-17T23:31:51.063703+0000","last_fullsized":"2024-09-17T23:31:51.063703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071413499999999997,"stat_sum":{"num_bytes":22429265,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":9609,"num_read_kb":34061,"num_write":1197,"num_write_kb":24063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":7449751,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.11","version":"146'350","reported_seq":8471,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.997682+0000","last_change":"2024-09-17T23:31:51.996353+0000","last_active":"2024-09-17T23:31:52.997682+0000","last_peered":"2024-09-17T23:31:52.997682+0000","last_clean":"2024-09-17T23:31:52.997682+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:31:52.997682+0000","last_undegraded":"2024-09-17T23:31:52.997682+0000","last_fullsized":"2024-09-17T23:31:52.997682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:45.874028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013198719999999999,"stat_sum":{"num_bytes":50975744,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":8068,"num_read_kb":46907,"num_write":328,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.12","version":"146'425","reported_seq":5724,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000779+0000","last_change":"2024-09-17T23:31:52.000145+0000","last_active":"2024-09-17T23:31:53.000779+0000","last_peered":"2024-09-17T23:31:53.000779+0000","last_clean":"2024-09-17T23:31:53.000779+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:31:53.000779+0000","last_undegraded":"2024-09-17T23:31:53.000779+0000","last_fullsized":"2024-09-17T23:31:53.000779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:17.892437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093631599999999997,"stat_sum":{"num_bytes":64148480,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":5018,"num_read_kb":38008,"num_write":478,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.13","version":"147'548","reported_seq":15089,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000840+0000","last_change":"2024-09-17T23:31:52.000994+0000","last_active":"2024-09-17T23:31:53.000840+0000","last_peered":"2024-09-17T23:31:53.000840+0000","last_clean":"2024-09-17T23:31:53.000840+0000","last_became_active":"2024-09-17T23:31:43.947677+0000","last_became_peered":"2024-09-17T23:31:43.947677+0000","last_unstale":"2024-09-17T23:31:53.000840+0000","last_undegraded":"2024-09-17T23:31:53.000840+0000","last_fullsized":"2024-09-17T23:31:53.000840+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:56.720660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001425017,"stat_sum":{"num_bytes":46287977,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":15068,"num_read_kb":74824,"num_write":703,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":797445174,"num_objects":925,"num_object_clones":202,"num_object_copies":1850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":407,"num_read":284475,"num_read_kb":1250598,"num_write":17611,"num_write_kb":466230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":224726827,"num_keys_recovered":888,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25411584,"data_stored":178587825,"data_compressed":9328555,"data_compressed_allocated":18665472,"data_compressed_original":174874624,"omap_allocated":7598,"internal_metadata":0},"log_size":15993,"ondisk_log_size":15993,"up":63,"acting":58,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500681,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48384,"kb_used_data":7488,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94323456,"statfs":{"total":96636764160,"available":96587218944,"internally_reserved":0,"allocated":7667712,"data_stored":41434757,"data_compressed":2009169,"data_compressed_allocated":4317184,"data_compressed_original":39789056,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68896,"kb_used_data":7180,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94302944,"statfs":{"total":96636764160,"available":96566214656,"internally_reserved":0,"allocated":7352320,"data_stored":39825151,"data_compressed":2155789,"data_compressed_allocated":4169728,"data_compressed_original":38337536,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78148,"kb_used_data":6704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293692,"statfs":{"total":96636764160,"available":96556740608,"internally_reserved":0,"allocated":6864896,"data_stored":39990186,"data_compressed":1832090,"data_compressed_allocated":3706880,"data_compressed_original":38471680,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640294,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640295,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76796,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295044,"statfs":{"total":96636764160,"available":96558125056,"internally_reserved":0,"allocated":5476352,"data_stored":27684937,"data_compressed":1281199,"data_compressed_allocated":2686976,"data_compressed_original":26363904,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3702784,"data_stored":26942103,"data_compressed":1281199,"data_compressed_allocated":2686976,"data_compressed_original":26363904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":2250800,"data_compressed":129518,"data_compressed_allocated":262144,"data_compressed_original":2220032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5095424,"data_stored":39247371,"data_compressed":1832090,"data_compressed_allocated":3706880,"data_compressed_original":38471680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5529600,"data_stored":38486272,"data_compressed":2150011,"data_compressed_allocated":4132864,"data_compressed_original":37747712,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5459968,"data_stored":37476179,"data_compressed":1949618,"data_compressed_allocated":3981312,"data_compressed_original":36663808,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:31:55.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:31:55.469 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:55.466+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:55.552 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:55.550+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:55.589 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:55.589 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:55.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:55.653 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:55.650+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:55.715 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:55.715 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:55.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:55.767 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:55.764+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:55.801 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:31:55.814 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-17T23:31:54.998526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":798035561,"num_objects":929,"num_object_clones":202,"num_object_copies":1858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":929,"num_whiteouts":407,"num_read":284591,"num_read_kb":1250698,"num_write":17772,"num_write_kb":468648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5297,"num_bytes_recovered":225317271,"num_keys_recovered":888,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16113,"ondisk_log_size":16113,"up":81,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":272224,"kb_used_data":26720,"kb_used_omap":24,"kb_used_meta":245351,"kb_avail":377215136,"statfs":{"total":386547056640,"available":386268299264,"internally_reserved":0,"allocated":27361280,"data_stored":148935031,"data_compressed":7278247,"data_compressed_allocated":14880768,"data_compressed_original":142962176,"omap_allocated":24578,"internal_metadata":251240446},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-138866866,"num_objects":-349,"num_object_clones":-134,"num_object_copies":-698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-349,"num_whiteouts":-146,"num_read":-23612,"num_read_kb":-71899,"num_write":-1260,"num_write_kb":-14423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-742,"num_bytes_recovered":-15921892,"num_keys_recovered":-222,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010629"},"pg_stats":[{"pgid":"3.1f","version":"146'413","reported_seq":5731,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:50.994093+0000","last_change":"2024-09-17T23:31:53.003682+0000","last_active":"2024-09-17T23:31:53.003682+0000","last_peered":"2024-09-17T23:31:50.994093+0000","last_clean":"2024-09-17T23:31:50.994093+0000","last_became_active":"2024-09-17T23:31:43.946390+0000","last_became_peered":"2024-09-17T23:31:43.946390+0000","last_unstale":"2024-09-17T23:31:50.994093+0000","last_undegraded":"2024-09-17T23:31:50.994093+0000","last_fullsized":"2024-09-17T23:31:50.994093+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001386211,"stat_sum":{"num_bytes":31087668,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":7434,"num_read_kb":66454,"num_write":453,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":21604086,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,5],"up_primary":3,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1e","version":"146'525","reported_seq":8881,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.999169+0000","last_change":"2024-09-17T23:31:51.996746+0000","last_active":"2024-09-17T23:31:52.999169+0000","last_peered":"2024-09-17T23:31:52.999169+0000","last_clean":"2024-09-17T23:31:52.999169+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:31:52.999169+0000","last_undegraded":"2024-09-17T23:31:52.999169+0000","last_fullsized":"2024-09-17T23:31:52.999169+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:29.234633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048813699999999998,"stat_sum":{"num_bytes":80398045,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":9314,"num_read_kb":55080,"num_write":535,"num_write_kb":19041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.1d","version":"146'345","reported_seq":5781,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001765+0000","last_change":"2024-09-17T23:31:52.000880+0000","last_active":"2024-09-17T23:31:53.001765+0000","last_peered":"2024-09-17T23:31:53.001765+0000","last_clean":"2024-09-17T23:31:53.001765+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:31:53.001765+0000","last_undegraded":"2024-09-17T23:31:53.001765+0000","last_fullsized":"2024-09-17T23:31:53.001765+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:37.132959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001352109,"stat_sum":{"num_bytes":16818688,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4805,"num_read_kb":41204,"num_write":320,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.1c","version":"147'428","reported_seq":9083,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.111570+0000","last_change":"2024-09-17T23:31:53.004165+0000","last_active":"2024-09-17T23:31:53.004165+0000","last_peered":"2024-09-17T23:31:51.111570+0000","last_clean":"2024-09-17T23:31:51.111570+0000","last_became_active":"2024-09-17T23:31:42.946855+0000","last_became_peered":"2024-09-17T23:31:42.946855+0000","last_unstale":"2024-09-17T23:31:51.111570+0000","last_undegraded":"2024-09-17T23:31:51.111570+0000","last_fullsized":"2024-09-17T23:31:51.111570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403729,"stat_sum":{"num_bytes":14477328,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":8439,"num_read_kb":40187,"num_write":516,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":14561502,"num_keys_recovered":350,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1b","version":"147'413","reported_seq":19288,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995904+0000","last_change":"2024-09-17T23:31:52.999783+0000","last_active":"2024-09-17T23:31:51.995904+0000","last_peered":"2024-09-17T23:31:51.138296+0000","last_clean":"2024-09-17T23:31:51.138296+0000","last_became_active":"2024-09-17T23:31:37.864076+0000","last_became_peered":"2024-09-17T23:31:37.864076+0000","last_unstale":"2024-09-17T23:31:51.995904+0000","last_undegraded":"2024-09-17T23:31:51.995904+0000","last_fullsized":"2024-09-17T23:31:51.995904+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062600000000000004,"stat_sum":{"num_bytes":62681915,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":18867,"num_read_kb":58412,"num_write":454,"num_write_kb":23019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":2836795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1a","version":"146'856","reported_seq":7400,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004831+0000","last_change":"2024-09-17T23:31:52.000256+0000","last_active":"2024-09-17T23:31:53.004831+0000","last_peered":"2024-09-17T23:31:53.004831+0000","last_clean":"2024-09-17T23:31:53.004831+0000","last_became_active":"2024-09-17T23:31:43.945744+0000","last_became_peered":"2024-09-17T23:31:43.945744+0000","last_unstale":"2024-09-17T23:31:53.004831+0000","last_undegraded":"2024-09-17T23:31:53.004831+0000","last_fullsized":"2024-09-17T23:31:53.004831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:54.415718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000452776,"stat_sum":{"num_bytes":31380669,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":11922,"num_read_kb":30828,"num_write":1189,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.19","version":"146'803","reported_seq":6496,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995822+0000","last_change":"2024-09-17T23:31:52.999112+0000","last_active":"2024-09-17T23:31:51.995822+0000","last_peered":"2024-09-17T23:31:50.998697+0000","last_clean":"2024-09-17T23:31:50.998697+0000","last_became_active":"2024-09-17T23:31:42.945986+0000","last_became_peered":"2024-09-17T23:31:42.945986+0000","last_unstale":"2024-09-17T23:31:51.995822+0000","last_undegraded":"2024-09-17T23:31:51.995822+0000","last_fullsized":"2024-09-17T23:31:51.995822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088431689999999997,"stat_sum":{"num_bytes":15988255,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":8532,"num_read_kb":17539,"num_write":911,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":4472153,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.18","version":"146'435","reported_seq":4255,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.997643+0000","last_change":"2024-09-17T23:31:51.995873+0000","last_active":"2024-09-17T23:31:52.997643+0000","last_peered":"2024-09-17T23:31:52.997643+0000","last_clean":"2024-09-17T23:31:52.997643+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:31:52.997643+0000","last_undegraded":"2024-09-17T23:31:52.997643+0000","last_fullsized":"2024-09-17T23:31:52.997643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:59.943227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841602,"stat_sum":{"num_bytes":17261108,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":4986,"num_read_kb":22191,"num_write":433,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.17","version":"146'597","reported_seq":7899,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-17T23:31:50.026446+0000","last_change":"2024-09-17T23:31:50.028992+0000","last_active":"2024-09-17T23:31:50.026446+0000","last_peered":"2024-09-17T23:31:50.026446+0000","last_clean":"2024-09-17T23:31:50.026446+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:31:50.026446+0000","last_undegraded":"2024-09-17T23:31:50.026446+0000","last_fullsized":"2024-09-17T23:31:50.026446+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:58:45.677907+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.049582080000000001,"stat_sum":{"num_bytes":36516954,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":10694,"num_read_kb":28755,"num_write":514,"num_write_kb":13263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.16","version":"146'351","reported_seq":4259,"reported_epoch":146,"state":"active+clean+remapped","last_fresh":"2024-09-17T23:31:49.989903+0000","last_change":"2024-09-17T23:31:49.991057+0000","last_active":"2024-09-17T23:31:49.989903+0000","last_peered":"2024-09-17T23:31:49.989903+0000","last_clean":"2024-09-17T23:31:49.989903+0000","last_became_active":"2024-09-17T23:31:40.879305+0000","last_became_peered":"2024-09-17T23:31:40.879305+0000","last_unstale":"2024-09-17T23:31:49.989903+0000","last_undegraded":"2024-09-17T23:31:49.989903+0000","last_fullsized":"2024-09-17T23:31:49.989903+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:50.049462+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.010952870999999999,"stat_sum":{"num_bytes":14573104,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4431,"num_read_kb":31183,"num_write":358,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5911184,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,5","objects":27}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.15","version":"146'581","reported_seq":12749,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001840+0000","last_change":"2024-09-17T23:31:52.002608+0000","last_active":"2024-09-17T23:31:53.001840+0000","last_peered":"2024-09-17T23:31:53.001840+0000","last_clean":"2024-09-17T23:31:53.001840+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:31:53.001840+0000","last_undegraded":"2024-09-17T23:31:53.001840+0000","last_fullsized":"2024-09-17T23:31:53.001840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:40.342815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002366412,"stat_sum":{"num_bytes":27632680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":14519,"num_read_kb":31572,"num_write":615,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.14","version":"147'536","reported_seq":7847,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001410+0000","last_change":"2024-09-17T23:31:52.000543+0000","last_active":"2024-09-17T23:31:53.001410+0000","last_peered":"2024-09-17T23:31:53.001410+0000","last_clean":"2024-09-17T23:31:53.001410+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:31:53.001410+0000","last_undegraded":"2024-09-17T23:31:53.001410+0000","last_fullsized":"2024-09-17T23:31:53.001410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:56:23.071341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001202137,"stat_sum":{"num_bytes":13340257,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":8401,"num_read_kb":33555,"num_write":450,"num_write_kb":18362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001697+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:31:53.001697+0000","last_peered":"2024-09-17T23:31:53.001697+0000","last_clean":"2024-09-17T23:31:53.001697+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:31:53.001697+0000","last_undegraded":"2024-09-17T23:31:53.001697+0000","last_fullsized":"2024-09-17T23:31:53.001697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"147'602","reported_seq":5017,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003390+0000","last_change":"2024-09-17T23:31:51.999436+0000","last_active":"2024-09-17T23:31:53.003390+0000","last_peered":"2024-09-17T23:31:53.003390+0000","last_clean":"2024-09-17T23:31:53.003390+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:31:53.003390+0000","last_undegraded":"2024-09-17T23:31:53.003390+0000","last_fullsized":"2024-09-17T23:31:53.003390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:43.562801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052716000000000004,"stat_sum":{"num_bytes":13084247,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5974,"num_read_kb":22618,"num_write":871,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":282,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003524+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:31:53.003524+0000","last_peered":"2024-09-17T23:31:53.003524+0000","last_clean":"2024-09-17T23:31:53.003524+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:31:53.003524+0000","last_undegraded":"2024-09-17T23:31:53.003524+0000","last_fullsized":"2024-09-17T23:31:53.003524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"147'414","reported_seq":12448,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.005411+0000","last_change":"2024-09-17T23:31:52.002346+0000","last_active":"2024-09-17T23:31:53.005411+0000","last_peered":"2024-09-17T23:31:53.005411+0000","last_clean":"2024-09-17T23:31:53.005411+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:31:53.005411+0000","last_undegraded":"2024-09-17T23:31:53.005411+0000","last_fullsized":"2024-09-17T23:31:53.005411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:38.208779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056097299999999998,"stat_sum":{"num_bytes":15504427,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":13024,"num_read_kb":60475,"num_write":417,"num_write_kb":30569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003656+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:31:53.003656+0000","last_peered":"2024-09-17T23:31:53.003656+0000","last_clean":"2024-09-17T23:31:53.003656+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:31:53.003656+0000","last_undegraded":"2024-09-17T23:31:53.003656+0000","last_fullsized":"2024-09-17T23:31:53.003656+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"146'381","reported_seq":7206,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004920+0000","last_change":"2024-09-17T23:31:52.001130+0000","last_active":"2024-09-17T23:31:53.004920+0000","last_peered":"2024-09-17T23:31:53.004920+0000","last_clean":"2024-09-17T23:31:53.004920+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:31:53.004920+0000","last_undegraded":"2024-09-17T23:31:53.004920+0000","last_fullsized":"2024-09-17T23:31:53.004920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:01.665826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078735100000000004,"stat_sum":{"num_bytes":14529566,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":4593,"num_read_kb":19709,"num_write":432,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":2,"reported_epoch":149,"state":"peering","last_fresh":"2024-09-17T23:31:52.000114+0000","last_change":"2024-09-17T23:31:53.398866+0000","last_active":"2024-09-17T23:31:53.398866+0000","last_peered":"2024-09-17T23:31:52.000114+0000","last_clean":"2024-09-17T23:31:52.000114+0000","last_became_active":"2024-09-17T23:31:33.918555+0000","last_became_peered":"2024-09-17T23:31:33.918555+0000","last_unstale":"2024-09-17T23:31:52.000114+0000","last_undegraded":"2024-09-17T23:31:52.000114+0000","last_fullsized":"2024-09-17T23:31:52.000114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"147'370","reported_seq":10454,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.998667+0000","last_change":"2024-09-17T23:31:51.996713+0000","last_active":"2024-09-17T23:31:52.998667+0000","last_peered":"2024-09-17T23:31:52.998667+0000","last_clean":"2024-09-17T23:31:52.998667+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:31:52.998667+0000","last_undegraded":"2024-09-17T23:31:52.998667+0000","last_fullsized":"2024-09-17T23:31:52.998667+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:59.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012702,"stat_sum":{"num_bytes":20594208,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":10196,"num_read_kb":27182,"num_write":428,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.3","version":"147'559","reported_seq":7277,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001255+0000","last_change":"2024-09-17T23:31:52.001784+0000","last_active":"2024-09-17T23:31:53.001255+0000","last_peered":"2024-09-17T23:31:53.001255+0000","last_clean":"2024-09-17T23:31:53.001255+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:31:53.001255+0000","last_undegraded":"2024-09-17T23:31:53.001255+0000","last_fullsized":"2024-09-17T23:31:53.001255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:24.363207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020241600000000001,"stat_sum":{"num_bytes":19790336,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":7259,"num_read_kb":29719,"num_write":580,"num_write_kb":9147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.2","version":"18'2","reported_seq":327,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000975+0000","last_change":"2024-09-17T23:31:39.878887+0000","last_active":"2024-09-17T23:31:53.000975+0000","last_peered":"2024-09-17T23:31:53.000975+0000","last_clean":"2024-09-17T23:31:53.000975+0000","last_became_active":"2024-09-17T23:31:39.873374+0000","last_became_peered":"2024-09-17T23:31:39.873374+0000","last_unstale":"2024-09-17T23:31:53.000975+0000","last_undegraded":"2024-09-17T23:31:53.000975+0000","last_fullsized":"2024-09-17T23:31:53.000975+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:49.977116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"unknown","last_fresh":"2024-09-17T23:31:52.999124+0000","last_change":"2024-09-17T23:31:53.398793+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:31:52.999124+0000","last_undegraded":"2024-09-17T23:31:52.999124+0000","last_fullsized":"2024-09-17T23:31:52.999124+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":287,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.998563+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:31:52.998563+0000","last_peered":"2024-09-17T23:31:52.998563+0000","last_clean":"2024-09-17T23:31:52.998563+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:31:52.998563+0000","last_undegraded":"2024-09-17T23:31:52.998563+0000","last_fullsized":"2024-09-17T23:31:52.998563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"unknown","last_fresh":"2024-09-17T23:31:53.000225+0000","last_change":"2024-09-17T23:31:53.398831+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:31:53.000225+0000","last_undegraded":"2024-09-17T23:31:53.000225+0000","last_fullsized":"2024-09-17T23:31:53.000225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002579+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:31:53.002579+0000","last_peered":"2024-09-17T23:31:53.002579+0000","last_clean":"2024-09-17T23:31:53.002579+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:31:53.002579+0000","last_undegraded":"2024-09-17T23:31:53.002579+0000","last_fullsized":"2024-09-17T23:31:53.002579+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"147'670","reported_seq":10958,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995561+0000","last_change":"2024-09-17T23:31:52.999594+0000","last_active":"2024-09-17T23:31:51.995561+0000","last_peered":"2024-09-17T23:31:50.999183+0000","last_clean":"2024-09-17T23:31:50.999183+0000","last_became_active":"2024-09-17T23:31:43.946105+0000","last_became_peered":"2024-09-17T23:31:43.946105+0000","last_unstale":"2024-09-17T23:31:51.995561+0000","last_undegraded":"2024-09-17T23:31:51.995561+0000","last_fullsized":"2024-09-17T23:31:51.995561+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0095413040000000005,"stat_sum":{"num_bytes":23587937,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11181,"num_read_kb":79280,"num_write":755,"num_write_kb":59941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15860611,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"2.3","version":"16'1","reported_seq":205,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004921+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:31:53.004921+0000","last_peered":"2024-09-17T23:31:53.004921+0000","last_clean":"2024-09-17T23:31:53.004921+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:31:53.004921+0000","last_undegraded":"2024-09-17T23:31:53.004921+0000","last_fullsized":"2024-09-17T23:31:53.004921+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002710+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:31:53.002710+0000","last_peered":"2024-09-17T23:31:53.002710+0000","last_clean":"2024-09-17T23:31:53.002710+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:31:53.002710+0000","last_undegraded":"2024-09-17T23:31:53.002710+0000","last_fullsized":"2024-09-17T23:31:53.002710+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"146'430","reported_seq":14649,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002926+0000","last_change":"2024-09-17T23:31:51.999823+0000","last_active":"2024-09-17T23:31:53.002926+0000","last_peered":"2024-09-17T23:31:53.002926+0000","last_clean":"2024-09-17T23:31:53.002926+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:31:53.002926+0000","last_undegraded":"2024-09-17T23:31:53.002926+0000","last_fullsized":"2024-09-17T23:31:53.002926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:12.145593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062045500000000001,"stat_sum":{"num_bytes":30823677,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":15144,"num_read_kb":73922,"num_write":381,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.9","version":"146'695","reported_seq":11364,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.104415+0000","last_change":"2024-09-17T23:31:52.998233+0000","last_active":"2024-09-17T23:31:52.998233+0000","last_peered":"2024-09-17T23:31:51.104415+0000","last_clean":"2024-09-17T23:31:51.104415+0000","last_became_active":"2024-09-17T23:31:36.863448+0000","last_became_peered":"2024-09-17T23:31:36.863448+0000","last_unstale":"2024-09-17T23:31:51.104415+0000","last_undegraded":"2024-09-17T23:31:51.104415+0000","last_fullsized":"2024-09-17T23:31:51.104415+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045313389999999997,"stat_sum":{"num_bytes":12758069,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":11139,"num_read_kb":39046,"num_write":811,"num_write_kb":25901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":1842284,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.a","version":"147'572","reported_seq":9639,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000786+0000","last_change":"2024-09-17T23:31:52.001891+0000","last_active":"2024-09-17T23:31:53.000786+0000","last_peered":"2024-09-17T23:31:53.000786+0000","last_clean":"2024-09-17T23:31:53.000786+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:31:53.000786+0000","last_undegraded":"2024-09-17T23:31:53.000786+0000","last_fullsized":"2024-09-17T23:31:53.000786+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:18.311799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023115129999999998,"stat_sum":{"num_bytes":12512903,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":10779,"num_read_kb":53142,"num_write":576,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.b","version":"147'892","reported_seq":11165,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.996707+0000","last_change":"2024-09-17T23:31:52.999472+0000","last_active":"2024-09-17T23:31:51.996707+0000","last_peered":"2024-09-17T23:31:51.136571+0000","last_clean":"2024-09-17T23:31:51.136571+0000","last_became_active":"2024-09-17T23:31:37.863905+0000","last_became_peered":"2024-09-17T23:31:37.863905+0000","last_unstale":"2024-09-17T23:31:51.996707+0000","last_undegraded":"2024-09-17T23:31:51.996707+0000","last_fullsized":"2024-09-17T23:31:51.996707+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003279213,"stat_sum":{"num_bytes":28522528,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":14913,"num_read_kb":61384,"num_write":876,"num_write_kb":11835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2676806,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.c","version":"147'457","reported_seq":5312,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001521+0000","last_change":"2024-09-17T23:31:52.001483+0000","last_active":"2024-09-17T23:31:53.001521+0000","last_peered":"2024-09-17T23:31:53.001521+0000","last_clean":"2024-09-17T23:31:53.001521+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:31:53.001521+0000","last_undegraded":"2024-09-17T23:31:53.001521+0000","last_fullsized":"2024-09-17T23:31:53.001521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:36.817924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017303850000000001,"stat_sum":{"num_bytes":9347775,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":5958,"num_read_kb":36790,"num_write":450,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.d","version":"147'407","reported_seq":9641,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001837+0000","last_change":"2024-09-17T23:31:52.002602+0000","last_active":"2024-09-17T23:31:53.001837+0000","last_peered":"2024-09-17T23:31:53.001837+0000","last_clean":"2024-09-17T23:31:53.001837+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:31:53.001837+0000","last_undegraded":"2024-09-17T23:31:53.001837+0000","last_fullsized":"2024-09-17T23:31:53.001837+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:23.128610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024799679999999999,"stat_sum":{"num_bytes":26479684,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":9068,"num_read_kb":42813,"num_write":454,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.e","version":"147'695","reported_seq":10003,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.005169+0000","last_change":"2024-09-17T23:31:52.999913+0000","last_active":"2024-09-17T23:31:52.999913+0000","last_peered":"2024-09-17T23:31:51.005169+0000","last_clean":"2024-09-17T23:31:51.005169+0000","last_became_active":"2024-09-17T23:31:38.869556+0000","last_became_peered":"2024-09-17T23:31:38.869556+0000","last_unstale":"2024-09-17T23:31:51.005169+0000","last_undegraded":"2024-09-17T23:31:51.005169+0000","last_fullsized":"2024-09-17T23:31:51.005169+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01512409,"stat_sum":{"num_bytes":17051340,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":10323,"num_read_kb":26354,"num_write":754,"num_write_kb":7190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":3804584,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.f","version":"146'357","reported_seq":4718,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:50.994722+0000","last_change":"2024-09-17T23:31:53.002972+0000","last_active":"2024-09-17T23:31:53.002972+0000","last_peered":"2024-09-17T23:31:50.994722+0000","last_clean":"2024-09-17T23:31:50.994722+0000","last_became_active":"2024-09-17T23:31:37.868909+0000","last_became_peered":"2024-09-17T23:31:37.868909+0000","last_unstale":"2024-09-17T23:31:50.994722+0000","last_undegraded":"2024-09-17T23:31:50.994722+0000","last_fullsized":"2024-09-17T23:31:50.994722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00159764,"stat_sum":{"num_bytes":6860340,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":4817,"num_read_kb":27404,"num_write":372,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":4115627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.10","version":"147'886","reported_seq":7769,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.063703+0000","last_change":"2024-09-17T23:31:52.998040+0000","last_active":"2024-09-17T23:31:52.998040+0000","last_peered":"2024-09-17T23:31:51.063703+0000","last_clean":"2024-09-17T23:31:51.063703+0000","last_became_active":"2024-09-17T23:31:43.944949+0000","last_became_peered":"2024-09-17T23:31:43.944949+0000","last_unstale":"2024-09-17T23:31:51.063703+0000","last_undegraded":"2024-09-17T23:31:51.063703+0000","last_fullsized":"2024-09-17T23:31:51.063703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071413499999999997,"stat_sum":{"num_bytes":22429265,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":9609,"num_read_kb":34061,"num_write":1197,"num_write_kb":24063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":7449751,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.11","version":"146'350","reported_seq":8471,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.997682+0000","last_change":"2024-09-17T23:31:51.996353+0000","last_active":"2024-09-17T23:31:52.997682+0000","last_peered":"2024-09-17T23:31:52.997682+0000","last_clean":"2024-09-17T23:31:52.997682+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:31:52.997682+0000","last_undegraded":"2024-09-17T23:31:52.997682+0000","last_fullsized":"2024-09-17T23:31:52.997682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:45.874028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013198719999999999,"stat_sum":{"num_bytes":50975744,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":8068,"num_read_kb":46907,"num_write":328,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.12","version":"146'425","reported_seq":5724,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000779+0000","last_change":"2024-09-17T23:31:52.000145+0000","last_active":"2024-09-17T23:31:53.000779+0000","last_peered":"2024-09-17T23:31:53.000779+0000","last_clean":"2024-09-17T23:31:53.000779+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:31:53.000779+0000","last_undegraded":"2024-09-17T23:31:53.000779+0000","last_fullsized":"2024-09-17T23:31:53.000779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:17.892437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093631599999999997,"stat_sum":{"num_bytes":64148480,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":5018,"num_read_kb":38008,"num_write":478,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.13","version":"147'548","reported_seq":15089,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000840+0000","last_change":"2024-09-17T23:31:52.000994+0000","last_active":"2024-09-17T23:31:53.000840+0000","last_peered":"2024-09-17T23:31:53.000840+0000","last_clean":"2024-09-17T23:31:53.000840+0000","last_became_active":"2024-09-17T23:31:43.947677+0000","last_became_peered":"2024-09-17T23:31:43.947677+0000","last_unstale":"2024-09-17T23:31:53.000840+0000","last_undegraded":"2024-09-17T23:31:53.000840+0000","last_fullsized":"2024-09-17T23:31:53.000840+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:56.720660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001425017,"stat_sum":{"num_bytes":46287977,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":15068,"num_read_kb":74824,"num_write":703,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":797445174,"num_objects":925,"num_object_clones":202,"num_object_copies":1850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":407,"num_read":284475,"num_read_kb":1250598,"num_write":17611,"num_write_kb":466230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":224726827,"num_keys_recovered":888,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25411584,"data_stored":178587825,"data_compressed":9328555,"data_compressed_allocated":18665472,"data_compressed_original":174874624,"omap_allocated":7598,"internal_metadata":0},"log_size":15993,"ondisk_log_size":15993,"up":63,"acting":58,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500681,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48384,"kb_used_data":7488,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94323456,"statfs":{"total":96636764160,"available":96587218944,"internally_reserved":0,"allocated":7667712,"data_stored":41434757,"data_compressed":2009169,"data_compressed_allocated":4317184,"data_compressed_original":39789056,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68896,"kb_used_data":7180,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94302944,"statfs":{"total":96636764160,"available":96566214656,"internally_reserved":0,"allocated":7352320,"data_stored":39825151,"data_compressed":2155789,"data_compressed_allocated":4169728,"data_compressed_original":38337536,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78148,"kb_used_data":6704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293692,"statfs":{"total":96636764160,"available":96556740608,"internally_reserved":0,"allocated":6864896,"data_stored":39990186,"data_compressed":1832090,"data_compressed_allocated":3706880,"data_compressed_original":38471680,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640294,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640295,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76796,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295044,"statfs":{"total":96636764160,"available":96558125056,"internally_reserved":0,"allocated":5476352,"data_stored":27684937,"data_compressed":1281199,"data_compressed_allocated":2686976,"data_compressed_original":26363904,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3702784,"data_stored":26942103,"data_compressed":1281199,"data_compressed_allocated":2686976,"data_compressed_original":26363904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":2250800,"data_compressed":129518,"data_compressed_allocated":262144,"data_compressed_original":2220032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5095424,"data_stored":39247371,"data_compressed":1832090,"data_compressed_allocated":3706880,"data_compressed_original":38471680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5529600,"data_stored":38486272,"data_compressed":2150011,"data_compressed_allocated":4132864,"data_compressed_original":37747712,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5459968,"data_stored":37476179,"data_compressed":1949618,"data_compressed_allocated":3981312,"data_compressed_original":36663808,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:31:55.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T23:31:55.855 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:55.852+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:55.883 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:55.883 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:55.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:55.978 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:55.976+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:56.055 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:56.056 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:56.053+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:56.157 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:56.154+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:56.205 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:56.205 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:56.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:56.226 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:56.227 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":190,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":152,"num_osds":6,"num_up_osds":6,"osd_up_since":1726615871,"num_in_osds":4,"osd_in_since":1726615898,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":27},{"state_name":"remapped+peering","count":10},{"state_name":"unknown","count":2},{"state_name":"active+clean+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":929,"data_bytes":798035561,"bytes_used":278757376,"bytes_avail":386268299264,"bytes_total":386547056640,"unknown_pgs_ratio":0.048780485987663269,"inactive_pgs_ratio":0.26829269528388977,"misplaced_objects":27,"misplaced_total":1858,"misplaced_ratio":0.014531754574811625},"fsmap":{"epoch":1,"btime":"2024-09-17T23:28:46:047555+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-17T23:30:00.370484+0000","services":{}},"progress_events":{}} 2024-09-17T23:31:56.240 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T23:31:56.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:31:56.259 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:56.256+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:56.323 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:56.323 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:56.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:56.360 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:56.357+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:56.461 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:56.458+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:56.481 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:56.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:56.562 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:56.559+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:56.572 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:31:56.582 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-17T23:31:56.391177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":797773417,"num_objects":926,"num_object_clones":201,"num_object_copies":1852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":926,"num_whiteouts":406,"num_read":284759,"num_read_kb":1250844,"num_write":17789,"num_write_kb":468654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5297,"num_bytes_recovered":225317271,"num_keys_recovered":888,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16137,"ondisk_log_size":16137,"up":81,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":271708,"kb_used_data":26248,"kb_used_omap":24,"kb_used_meta":245351,"kb_avail":377215652,"statfs":{"total":386547056640,"available":386268827648,"internally_reserved":0,"allocated":26877952,"data_stored":145852217,"data_compressed":7163276,"data_compressed_allocated":14487552,"data_compressed_original":139955712,"omap_allocated":24578,"internal_metadata":251240446},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-139129010,"num_objects":-352,"num_object_clones":-135,"num_object_copies":-704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-352,"num_whiteouts":-147,"num_read":-23444,"num_read_kb":-71753,"num_write":-1243,"num_write_kb":-14417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-742,"num_bytes_recovered":-15921892,"num_keys_recovered":-222,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000727"},"pg_stats":[{"pgid":"3.1f","version":"146'413","reported_seq":5731,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:50.994093+0000","last_change":"2024-09-17T23:31:53.003682+0000","last_active":"2024-09-17T23:31:53.003682+0000","last_peered":"2024-09-17T23:31:50.994093+0000","last_clean":"2024-09-17T23:31:50.994093+0000","last_became_active":"2024-09-17T23:31:43.946390+0000","last_became_peered":"2024-09-17T23:31:43.946390+0000","last_unstale":"2024-09-17T23:31:50.994093+0000","last_undegraded":"2024-09-17T23:31:50.994093+0000","last_fullsized":"2024-09-17T23:31:50.994093+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001386211,"stat_sum":{"num_bytes":31087668,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":7434,"num_read_kb":66454,"num_write":453,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":21604086,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,5],"up_primary":3,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1e","version":"146'525","reported_seq":8881,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.999169+0000","last_change":"2024-09-17T23:31:51.996746+0000","last_active":"2024-09-17T23:31:52.999169+0000","last_peered":"2024-09-17T23:31:52.999169+0000","last_clean":"2024-09-17T23:31:52.999169+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:31:52.999169+0000","last_undegraded":"2024-09-17T23:31:52.999169+0000","last_fullsized":"2024-09-17T23:31:52.999169+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:29.234633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048813699999999998,"stat_sum":{"num_bytes":80398045,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":9314,"num_read_kb":55080,"num_write":535,"num_write_kb":19041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.1d","version":"146'345","reported_seq":5781,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001765+0000","last_change":"2024-09-17T23:31:52.000880+0000","last_active":"2024-09-17T23:31:53.001765+0000","last_peered":"2024-09-17T23:31:53.001765+0000","last_clean":"2024-09-17T23:31:53.001765+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:31:53.001765+0000","last_undegraded":"2024-09-17T23:31:53.001765+0000","last_fullsized":"2024-09-17T23:31:53.001765+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:37.132959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001352109,"stat_sum":{"num_bytes":16818688,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":4805,"num_read_kb":41204,"num_write":320,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.1c","version":"147'428","reported_seq":9083,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.111570+0000","last_change":"2024-09-17T23:31:53.004165+0000","last_active":"2024-09-17T23:31:53.004165+0000","last_peered":"2024-09-17T23:31:51.111570+0000","last_clean":"2024-09-17T23:31:51.111570+0000","last_became_active":"2024-09-17T23:31:42.946855+0000","last_became_peered":"2024-09-17T23:31:42.946855+0000","last_unstale":"2024-09-17T23:31:51.111570+0000","last_undegraded":"2024-09-17T23:31:51.111570+0000","last_fullsized":"2024-09-17T23:31:51.111570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001403729,"stat_sum":{"num_bytes":14477328,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":8439,"num_read_kb":40187,"num_write":516,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":14561502,"num_keys_recovered":350,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1b","version":"147'413","reported_seq":19288,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995904+0000","last_change":"2024-09-17T23:31:52.999783+0000","last_active":"2024-09-17T23:31:51.995904+0000","last_peered":"2024-09-17T23:31:51.138296+0000","last_clean":"2024-09-17T23:31:51.138296+0000","last_became_active":"2024-09-17T23:31:37.864076+0000","last_became_peered":"2024-09-17T23:31:37.864076+0000","last_unstale":"2024-09-17T23:31:51.995904+0000","last_undegraded":"2024-09-17T23:31:51.995904+0000","last_fullsized":"2024-09-17T23:31:51.995904+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062600000000000004,"stat_sum":{"num_bytes":62681915,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":22,"num_read":18867,"num_read_kb":58412,"num_write":454,"num_write_kb":23019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":2836795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.1a","version":"146'856","reported_seq":7400,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004831+0000","last_change":"2024-09-17T23:31:52.000256+0000","last_active":"2024-09-17T23:31:53.004831+0000","last_peered":"2024-09-17T23:31:53.004831+0000","last_clean":"2024-09-17T23:31:53.004831+0000","last_became_active":"2024-09-17T23:31:43.945744+0000","last_became_peered":"2024-09-17T23:31:43.945744+0000","last_unstale":"2024-09-17T23:31:53.004831+0000","last_undegraded":"2024-09-17T23:31:53.004831+0000","last_fullsized":"2024-09-17T23:31:53.004831+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:54.415718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000452776,"stat_sum":{"num_bytes":31380669,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":11922,"num_read_kb":30828,"num_write":1189,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.19","version":"146'803","reported_seq":6496,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995822+0000","last_change":"2024-09-17T23:31:52.999112+0000","last_active":"2024-09-17T23:31:51.995822+0000","last_peered":"2024-09-17T23:31:50.998697+0000","last_clean":"2024-09-17T23:31:50.998697+0000","last_became_active":"2024-09-17T23:31:42.945986+0000","last_became_peered":"2024-09-17T23:31:42.945986+0000","last_unstale":"2024-09-17T23:31:51.995822+0000","last_undegraded":"2024-09-17T23:31:51.995822+0000","last_fullsized":"2024-09-17T23:31:51.995822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088431689999999997,"stat_sum":{"num_bytes":15988255,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":8532,"num_read_kb":17539,"num_write":911,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":4472153,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.18","version":"146'435","reported_seq":4255,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.997643+0000","last_change":"2024-09-17T23:31:51.995873+0000","last_active":"2024-09-17T23:31:52.997643+0000","last_peered":"2024-09-17T23:31:52.997643+0000","last_clean":"2024-09-17T23:31:52.997643+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:31:52.997643+0000","last_undegraded":"2024-09-17T23:31:52.997643+0000","last_fullsized":"2024-09-17T23:31:52.997643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:59.943227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000841602,"stat_sum":{"num_bytes":17261108,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":4986,"num_read_kb":22191,"num_write":433,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.17","version":"147'621","reported_seq":8014,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T23:31:55.003221+0000","last_change":"2024-09-17T23:31:52.002603+0000","last_active":"2024-09-17T23:31:55.003221+0000","last_peered":"2024-09-17T23:31:55.003221+0000","last_clean":"2024-09-17T23:31:55.003221+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:31:55.003221+0000","last_undegraded":"2024-09-17T23:31:55.003221+0000","last_fullsized":"2024-09-17T23:31:55.003221+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:24.049727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010706089999999999,"stat_sum":{"num_bytes":36254810,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":10862,"num_read_kb":28901,"num_write":531,"num_write_kb":13269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.16","version":"146'351","reported_seq":4259,"reported_epoch":146,"state":"active+clean+remapped","last_fresh":"2024-09-17T23:31:49.989903+0000","last_change":"2024-09-17T23:31:49.991057+0000","last_active":"2024-09-17T23:31:49.989903+0000","last_peered":"2024-09-17T23:31:49.989903+0000","last_clean":"2024-09-17T23:31:49.989903+0000","last_became_active":"2024-09-17T23:31:40.879305+0000","last_became_peered":"2024-09-17T23:31:40.879305+0000","last_unstale":"2024-09-17T23:31:49.989903+0000","last_undegraded":"2024-09-17T23:31:49.989903+0000","last_fullsized":"2024-09-17T23:31:49.989903+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:50.049462+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.010952870999999999,"stat_sum":{"num_bytes":14573104,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4431,"num_read_kb":31183,"num_write":358,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":5911184,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,5","objects":27}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"8"}]},{"pgid":"3.15","version":"146'581","reported_seq":12749,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001840+0000","last_change":"2024-09-17T23:31:52.002608+0000","last_active":"2024-09-17T23:31:53.001840+0000","last_peered":"2024-09-17T23:31:53.001840+0000","last_clean":"2024-09-17T23:31:53.001840+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:31:53.001840+0000","last_undegraded":"2024-09-17T23:31:53.001840+0000","last_fullsized":"2024-09-17T23:31:53.001840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:40.342815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002366412,"stat_sum":{"num_bytes":27632680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":14519,"num_read_kb":31572,"num_write":615,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.14","version":"147'536","reported_seq":7847,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001410+0000","last_change":"2024-09-17T23:31:52.000543+0000","last_active":"2024-09-17T23:31:53.001410+0000","last_peered":"2024-09-17T23:31:53.001410+0000","last_clean":"2024-09-17T23:31:53.001410+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:31:53.001410+0000","last_undegraded":"2024-09-17T23:31:53.001410+0000","last_fullsized":"2024-09-17T23:31:53.001410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:56:23.071341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001202137,"stat_sum":{"num_bytes":13340257,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":11,"num_read":8401,"num_read_kb":33555,"num_write":450,"num_write_kb":18362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001697+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:31:53.001697+0000","last_peered":"2024-09-17T23:31:53.001697+0000","last_clean":"2024-09-17T23:31:53.001697+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:31:53.001697+0000","last_undegraded":"2024-09-17T23:31:53.001697+0000","last_fullsized":"2024-09-17T23:31:53.001697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"147'602","reported_seq":5017,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003390+0000","last_change":"2024-09-17T23:31:51.999436+0000","last_active":"2024-09-17T23:31:53.003390+0000","last_peered":"2024-09-17T23:31:53.003390+0000","last_clean":"2024-09-17T23:31:53.003390+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:31:53.003390+0000","last_undegraded":"2024-09-17T23:31:53.003390+0000","last_fullsized":"2024-09-17T23:31:53.003390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:43.562801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052716000000000004,"stat_sum":{"num_bytes":13084247,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":11,"num_read":5974,"num_read_kb":22618,"num_write":871,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":282,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003524+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:31:53.003524+0000","last_peered":"2024-09-17T23:31:53.003524+0000","last_clean":"2024-09-17T23:31:53.003524+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:31:53.003524+0000","last_undegraded":"2024-09-17T23:31:53.003524+0000","last_fullsized":"2024-09-17T23:31:53.003524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"147'414","reported_seq":12448,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.005411+0000","last_change":"2024-09-17T23:31:52.002346+0000","last_active":"2024-09-17T23:31:53.005411+0000","last_peered":"2024-09-17T23:31:53.005411+0000","last_clean":"2024-09-17T23:31:53.005411+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:31:53.005411+0000","last_undegraded":"2024-09-17T23:31:53.005411+0000","last_fullsized":"2024-09-17T23:31:53.005411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:38.208779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056097299999999998,"stat_sum":{"num_bytes":15504427,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":13024,"num_read_kb":60475,"num_write":417,"num_write_kb":30569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.003656+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:31:53.003656+0000","last_peered":"2024-09-17T23:31:53.003656+0000","last_clean":"2024-09-17T23:31:53.003656+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:31:53.003656+0000","last_undegraded":"2024-09-17T23:31:53.003656+0000","last_fullsized":"2024-09-17T23:31:53.003656+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"146'381","reported_seq":7206,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004920+0000","last_change":"2024-09-17T23:31:52.001130+0000","last_active":"2024-09-17T23:31:53.004920+0000","last_peered":"2024-09-17T23:31:53.004920+0000","last_clean":"2024-09-17T23:31:53.004920+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:31:53.004920+0000","last_undegraded":"2024-09-17T23:31:53.004920+0000","last_fullsized":"2024-09-17T23:31:53.004920+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:01.665826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078735100000000004,"stat_sum":{"num_bytes":14529566,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":4593,"num_read_kb":19709,"num_write":432,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":2,"reported_epoch":149,"state":"peering","last_fresh":"2024-09-17T23:31:52.000114+0000","last_change":"2024-09-17T23:31:53.398866+0000","last_active":"2024-09-17T23:31:53.398866+0000","last_peered":"2024-09-17T23:31:52.000114+0000","last_clean":"2024-09-17T23:31:52.000114+0000","last_became_active":"2024-09-17T23:31:33.918555+0000","last_became_peered":"2024-09-17T23:31:33.918555+0000","last_unstale":"2024-09-17T23:31:52.000114+0000","last_undegraded":"2024-09-17T23:31:52.000114+0000","last_fullsized":"2024-09-17T23:31:52.000114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"147'370","reported_seq":10454,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.998667+0000","last_change":"2024-09-17T23:31:51.996713+0000","last_active":"2024-09-17T23:31:52.998667+0000","last_peered":"2024-09-17T23:31:52.998667+0000","last_clean":"2024-09-17T23:31:52.998667+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:31:52.998667+0000","last_undegraded":"2024-09-17T23:31:52.998667+0000","last_fullsized":"2024-09-17T23:31:52.998667+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:59.398637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012702,"stat_sum":{"num_bytes":20594208,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":10196,"num_read_kb":27182,"num_write":428,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.3","version":"147'559","reported_seq":7277,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001255+0000","last_change":"2024-09-17T23:31:52.001784+0000","last_active":"2024-09-17T23:31:53.001255+0000","last_peered":"2024-09-17T23:31:53.001255+0000","last_clean":"2024-09-17T23:31:53.001255+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:31:53.001255+0000","last_undegraded":"2024-09-17T23:31:53.001255+0000","last_fullsized":"2024-09-17T23:31:53.001255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:24.363207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020241600000000001,"stat_sum":{"num_bytes":19790336,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":24,"num_read":7259,"num_read_kb":29719,"num_write":580,"num_write_kb":9147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"2.2","version":"18'2","reported_seq":327,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000975+0000","last_change":"2024-09-17T23:31:39.878887+0000","last_active":"2024-09-17T23:31:53.000975+0000","last_peered":"2024-09-17T23:31:53.000975+0000","last_clean":"2024-09-17T23:31:53.000975+0000","last_became_active":"2024-09-17T23:31:39.873374+0000","last_became_peered":"2024-09-17T23:31:39.873374+0000","last_unstale":"2024-09-17T23:31:53.000975+0000","last_undegraded":"2024-09-17T23:31:53.000975+0000","last_fullsized":"2024-09-17T23:31:53.000975+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:49.977116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"unknown","last_fresh":"2024-09-17T23:31:52.999124+0000","last_change":"2024-09-17T23:31:53.398793+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:31:52.999124+0000","last_undegraded":"2024-09-17T23:31:52.999124+0000","last_fullsized":"2024-09-17T23:31:52.999124+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":287,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.998563+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:31:52.998563+0000","last_peered":"2024-09-17T23:31:52.998563+0000","last_clean":"2024-09-17T23:31:52.998563+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:31:52.998563+0000","last_undegraded":"2024-09-17T23:31:52.998563+0000","last_fullsized":"2024-09-17T23:31:52.998563+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"unknown","last_fresh":"2024-09-17T23:31:53.000225+0000","last_change":"2024-09-17T23:31:53.398831+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T23:31:53.000225+0000","last_undegraded":"2024-09-17T23:31:53.000225+0000","last_fullsized":"2024-09-17T23:31:53.000225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002579+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:31:53.002579+0000","last_peered":"2024-09-17T23:31:53.002579+0000","last_clean":"2024-09-17T23:31:53.002579+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:31:53.002579+0000","last_undegraded":"2024-09-17T23:31:53.002579+0000","last_fullsized":"2024-09-17T23:31:53.002579+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"147'670","reported_seq":10958,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.995561+0000","last_change":"2024-09-17T23:31:52.999594+0000","last_active":"2024-09-17T23:31:51.995561+0000","last_peered":"2024-09-17T23:31:50.999183+0000","last_clean":"2024-09-17T23:31:50.999183+0000","last_became_active":"2024-09-17T23:31:43.946105+0000","last_became_peered":"2024-09-17T23:31:43.946105+0000","last_unstale":"2024-09-17T23:31:51.995561+0000","last_undegraded":"2024-09-17T23:31:51.995561+0000","last_fullsized":"2024-09-17T23:31:51.995561+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0095413040000000005,"stat_sum":{"num_bytes":23587937,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11181,"num_read_kb":79280,"num_write":755,"num_write_kb":59941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":15860611,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"2.3","version":"16'1","reported_seq":205,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.004921+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:31:53.004921+0000","last_peered":"2024-09-17T23:31:53.004921+0000","last_clean":"2024-09-17T23:31:53.004921+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:31:53.004921+0000","last_undegraded":"2024-09-17T23:31:53.004921+0000","last_fullsized":"2024-09-17T23:31:53.004921+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":371,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002710+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:31:53.002710+0000","last_peered":"2024-09-17T23:31:53.002710+0000","last_clean":"2024-09-17T23:31:53.002710+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:31:53.002710+0000","last_undegraded":"2024-09-17T23:31:53.002710+0000","last_fullsized":"2024-09-17T23:31:53.002710+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"146'430","reported_seq":14649,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.002926+0000","last_change":"2024-09-17T23:31:51.999823+0000","last_active":"2024-09-17T23:31:53.002926+0000","last_peered":"2024-09-17T23:31:53.002926+0000","last_clean":"2024-09-17T23:31:53.002926+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:31:53.002926+0000","last_undegraded":"2024-09-17T23:31:53.002926+0000","last_fullsized":"2024-09-17T23:31:53.002926+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:12.145593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062045500000000001,"stat_sum":{"num_bytes":30823677,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":15144,"num_read_kb":73922,"num_write":381,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.9","version":"146'695","reported_seq":11364,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.104415+0000","last_change":"2024-09-17T23:31:52.998233+0000","last_active":"2024-09-17T23:31:52.998233+0000","last_peered":"2024-09-17T23:31:51.104415+0000","last_clean":"2024-09-17T23:31:51.104415+0000","last_became_active":"2024-09-17T23:31:36.863448+0000","last_became_peered":"2024-09-17T23:31:36.863448+0000","last_unstale":"2024-09-17T23:31:51.104415+0000","last_undegraded":"2024-09-17T23:31:51.104415+0000","last_fullsized":"2024-09-17T23:31:51.104415+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045313389999999997,"stat_sum":{"num_bytes":12758069,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":11139,"num_read_kb":39046,"num_write":811,"num_write_kb":25901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":1842284,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.a","version":"147'572","reported_seq":9639,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000786+0000","last_change":"2024-09-17T23:31:52.001891+0000","last_active":"2024-09-17T23:31:53.000786+0000","last_peered":"2024-09-17T23:31:53.000786+0000","last_clean":"2024-09-17T23:31:53.000786+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:31:53.000786+0000","last_undegraded":"2024-09-17T23:31:53.000786+0000","last_fullsized":"2024-09-17T23:31:53.000786+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:18.311799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023115129999999998,"stat_sum":{"num_bytes":12512903,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":16,"num_read":10779,"num_read_kb":53142,"num_write":576,"num_write_kb":7234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.b","version":"147'892","reported_seq":11165,"reported_epoch":148,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.996707+0000","last_change":"2024-09-17T23:31:52.999472+0000","last_active":"2024-09-17T23:31:51.996707+0000","last_peered":"2024-09-17T23:31:51.136571+0000","last_clean":"2024-09-17T23:31:51.136571+0000","last_became_active":"2024-09-17T23:31:37.863905+0000","last_became_peered":"2024-09-17T23:31:37.863905+0000","last_unstale":"2024-09-17T23:31:51.996707+0000","last_undegraded":"2024-09-17T23:31:51.996707+0000","last_fullsized":"2024-09-17T23:31:51.996707+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003279213,"stat_sum":{"num_bytes":28522528,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":14913,"num_read_kb":61384,"num_write":876,"num_write_kb":11835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2676806,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.c","version":"147'457","reported_seq":5312,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001521+0000","last_change":"2024-09-17T23:31:52.001483+0000","last_active":"2024-09-17T23:31:53.001521+0000","last_peered":"2024-09-17T23:31:53.001521+0000","last_clean":"2024-09-17T23:31:53.001521+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:31:53.001521+0000","last_undegraded":"2024-09-17T23:31:53.001521+0000","last_fullsized":"2024-09-17T23:31:53.001521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:36.817924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017303850000000001,"stat_sum":{"num_bytes":9347775,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":5958,"num_read_kb":36790,"num_write":450,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.d","version":"147'407","reported_seq":9641,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.001837+0000","last_change":"2024-09-17T23:31:52.002602+0000","last_active":"2024-09-17T23:31:53.001837+0000","last_peered":"2024-09-17T23:31:53.001837+0000","last_clean":"2024-09-17T23:31:53.001837+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:31:53.001837+0000","last_undegraded":"2024-09-17T23:31:53.001837+0000","last_fullsized":"2024-09-17T23:31:53.001837+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:23.128610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024799679999999999,"stat_sum":{"num_bytes":26479684,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":9068,"num_read_kb":42813,"num_write":454,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.e","version":"147'695","reported_seq":10003,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.005169+0000","last_change":"2024-09-17T23:31:52.999913+0000","last_active":"2024-09-17T23:31:52.999913+0000","last_peered":"2024-09-17T23:31:51.005169+0000","last_clean":"2024-09-17T23:31:51.005169+0000","last_became_active":"2024-09-17T23:31:38.869556+0000","last_became_peered":"2024-09-17T23:31:38.869556+0000","last_unstale":"2024-09-17T23:31:51.005169+0000","last_undegraded":"2024-09-17T23:31:51.005169+0000","last_fullsized":"2024-09-17T23:31:51.005169+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01512409,"stat_sum":{"num_bytes":17051340,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":10323,"num_read_kb":26354,"num_write":754,"num_write_kb":7190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":3804584,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.f","version":"146'357","reported_seq":4718,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:50.994722+0000","last_change":"2024-09-17T23:31:53.002972+0000","last_active":"2024-09-17T23:31:53.002972+0000","last_peered":"2024-09-17T23:31:50.994722+0000","last_clean":"2024-09-17T23:31:50.994722+0000","last_became_active":"2024-09-17T23:31:37.868909+0000","last_became_peered":"2024-09-17T23:31:37.868909+0000","last_unstale":"2024-09-17T23:31:50.994722+0000","last_undegraded":"2024-09-17T23:31:50.994722+0000","last_fullsized":"2024-09-17T23:31:50.994722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00159764,"stat_sum":{"num_bytes":6860340,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":4817,"num_read_kb":27404,"num_write":372,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":4115627,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":4,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.10","version":"147'886","reported_seq":7769,"reported_epoch":147,"state":"remapped+peering","last_fresh":"2024-09-17T23:31:51.063703+0000","last_change":"2024-09-17T23:31:52.998040+0000","last_active":"2024-09-17T23:31:52.998040+0000","last_peered":"2024-09-17T23:31:51.063703+0000","last_clean":"2024-09-17T23:31:51.063703+0000","last_became_active":"2024-09-17T23:31:43.944949+0000","last_became_peered":"2024-09-17T23:31:43.944949+0000","last_unstale":"2024-09-17T23:31:51.063703+0000","last_undegraded":"2024-09-17T23:31:51.063703+0000","last_fullsized":"2024-09-17T23:31:51.063703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071413499999999997,"stat_sum":{"num_bytes":22429265,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":9609,"num_read_kb":34061,"num_write":1197,"num_write_kb":24063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":7449751,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":0,"acting_primary":3,"purged_snaps":[{"start":"19","length":"9"}]},{"pgid":"3.11","version":"146'350","reported_seq":8471,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:52.997682+0000","last_change":"2024-09-17T23:31:51.996353+0000","last_active":"2024-09-17T23:31:52.997682+0000","last_peered":"2024-09-17T23:31:52.997682+0000","last_clean":"2024-09-17T23:31:52.997682+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:31:52.997682+0000","last_undegraded":"2024-09-17T23:31:52.997682+0000","last_fullsized":"2024-09-17T23:31:52.997682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:45.874028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013198719999999999,"stat_sum":{"num_bytes":50975744,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":8068,"num_read_kb":46907,"num_write":328,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.12","version":"146'425","reported_seq":5724,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000779+0000","last_change":"2024-09-17T23:31:52.000145+0000","last_active":"2024-09-17T23:31:53.000779+0000","last_peered":"2024-09-17T23:31:53.000779+0000","last_clean":"2024-09-17T23:31:53.000779+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:31:53.000779+0000","last_undegraded":"2024-09-17T23:31:53.000779+0000","last_fullsized":"2024-09-17T23:31:53.000779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:17.892437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093631599999999997,"stat_sum":{"num_bytes":64148480,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":5018,"num_read_kb":38008,"num_write":478,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.13","version":"147'548","reported_seq":15089,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T23:31:53.000840+0000","last_change":"2024-09-17T23:31:52.000994+0000","last_active":"2024-09-17T23:31:53.000840+0000","last_peered":"2024-09-17T23:31:53.000840+0000","last_clean":"2024-09-17T23:31:53.000840+0000","last_became_active":"2024-09-17T23:31:43.947677+0000","last_became_peered":"2024-09-17T23:31:43.947677+0000","last_unstale":"2024-09-17T23:31:53.000840+0000","last_undegraded":"2024-09-17T23:31:53.000840+0000","last_fullsized":"2024-09-17T23:31:53.000840+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:56.720660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001425017,"stat_sum":{"num_bytes":46287977,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":15068,"num_read_kb":74824,"num_write":703,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"18","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":797183030,"num_objects":922,"num_object_clones":201,"num_object_copies":1844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":27,"num_objects_unfound":0,"num_objects_dirty":922,"num_whiteouts":406,"num_read":284643,"num_read_kb":1250744,"num_write":17628,"num_write_kb":466236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":224726827,"num_keys_recovered":888,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25088000,"data_stored":177521716,"data_compressed":9202096,"data_compressed_allocated":18456576,"data_compressed_original":173867008,"omap_allocated":7598,"internal_metadata":0},"log_size":16017,"ondisk_log_size":16017,"up":63,"acting":58,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500682,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":7016,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":7184384,"data_stored":38351943,"data_compressed":1894198,"data_compressed_allocated":3923968,"data_compressed_original":36782592,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640294,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68896,"kb_used_data":7180,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94302944,"statfs":{"total":96636764160,"available":96566214656,"internally_reserved":0,"allocated":7352320,"data_stored":39825151,"data_compressed":2155789,"data_compressed_allocated":4169728,"data_compressed_original":38337536,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78148,"kb_used_data":6704,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293692,"statfs":{"total":96636764160,"available":96556740608,"internally_reserved":0,"allocated":6864896,"data_stored":39990186,"data_compressed":1832090,"data_compressed_allocated":3706880,"data_compressed_original":38471680,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640294,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640295,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76796,"kb_used_data":5348,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94295044,"statfs":{"total":96636764160,"available":96558125056,"internally_reserved":0,"allocated":5476352,"data_stored":27684937,"data_compressed":1281199,"data_compressed_allocated":2686976,"data_compressed_original":26363904,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3702784,"data_stored":26942103,"data_compressed":1281199,"data_compressed_allocated":2686976,"data_compressed_original":26363904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":2250800,"data_compressed":129518,"data_compressed_allocated":262144,"data_compressed_original":2220032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5095424,"data_stored":39247371,"data_compressed":1832090,"data_compressed_allocated":3706880,"data_compressed_original":38471680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5529600,"data_stored":38486272,"data_compressed":2150011,"data_compressed_allocated":4132864,"data_compressed_original":37747712,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5136384,"data_stored":36410070,"data_compressed":1823159,"data_compressed_allocated":3772416,"data_compressed_original":35656192,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:31:56.599 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:56.599 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:56.608 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:56.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:31:56.693 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:56.690+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:56.756 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:56.757 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:56.764 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:56.761+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:56.765 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:56.771 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:56.771 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:56.788 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:56.788 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:56.804 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:56.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:56.821 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:56.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:56.840 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:56.854 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:56.854 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:56.864 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:56.862+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:56.923 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:56.923 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.045676176, 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.925 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.926 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:31:56.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.03005104600001, 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:31:56.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:31:56.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.03001338999999, 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.936 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:31:56.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:31:56.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.02999595899999, 2024-09-17T23:31:56.940 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:31:56.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.943 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.944 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.945 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.029961028, 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:56.946 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:31:56.947 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.948 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:56.950 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.029929608, 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.951 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.952 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:31:56.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.954 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.955 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.02991298200001, 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.956 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.957 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:31:56.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:31:56.959 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.960 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.02989128799999, 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.962 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.963 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:31:56.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 150.02987956999999, 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:31:56.966 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.967 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:31:56.968 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.969 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.970 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:31:56.971 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.070741050999999, 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:56.972 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.973 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.974 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:31:56.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:31:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.977 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:31:56.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.979 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.066785886999995, 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.980 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.982 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.983 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:31:56.984 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.066745229000006, 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:56.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.987 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:31:56.988 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.989 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.990 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.066449582000004, 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.992 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:31:56.993 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.994 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.995 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.996 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.066417286999993, 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:56.997 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.998 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:31:56.999 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.000 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:31:57.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.002 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.066262476000006, 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.003 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.004 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:31:57.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.006 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.007 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 91.065976156000005, 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:31:57.008 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.010 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:31:57.012 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 91.065907495000005, 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:31:57.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.015 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.016 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:31:57.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 91.065865521000006, 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.020 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.021 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:31:57.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:31:57.023 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.066081498000003, 2024-09-17T23:31:57.024 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:57.025 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.026 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:31:57.028 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.029 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:24856 3.b 3:d3034e8c:::rbd_header.1152ea45e50a:head [watch reconnect cookie 140329332260848 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:31:57.030 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.9147621620000002, 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0448234409999999, 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 24856 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.031 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005662+0000", 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9350000000000001e-06 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005669+0000", 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3470000000000004e-06 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005683+0000", 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4419999999999999e-05 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005694+0000", 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0917999999999999e-05 2024-09-17T23:31:57.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006157+0000", 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00046328800000000001 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006166+0000", 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041545+0000", 2024-09-17T23:31:57.034 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.035378916 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041630+0000", 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5628000000000001e-05 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045500+0000", 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038699910000000001 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.035 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045553+0000", 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2516000000000003e-05 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.050484+0000", 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049310329999999996 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.036 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:57.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:31:57.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:57.039 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:57.035+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:57.138 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:57.135+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:57.187 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:57.188 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:57.188 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:57.196 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:31:57.239 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:57.236+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:57.340 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:57.337+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:57.343 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:57.343 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:57.343 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:57.343 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:57.343 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:57.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:31:57.441 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:57.438+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.004882401, 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.503 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:31:57.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.505 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.506 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00486252100001, 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.508 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:31:57.509 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.510 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:57.511 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00484098300001, 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.512 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.513 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:31:57.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.002948269, 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.517 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.518 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:31:57.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00293603899999, 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.521 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:31:57.522 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.523 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.524 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00292547000001, 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:31:57.525 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.526 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:31:57.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.528 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00291558200001, 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:31:57.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.532 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:31:57.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00290530000001, 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.536 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:31:57.537 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.538 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:31:57.539 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 140.00288941299999, 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.540 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:31:57.541 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.542 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.543 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.463964096, 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:31:57.544 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.545 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:31:57.546 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.547 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:31:57.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:31:57.549 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 119.46189471, 2024-09-17T23:31:57.550 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.552 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.553 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:31:57.554 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.555 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:31:57.556 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 118.13002734200001, 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.557 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:31:57.559 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.560 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.561 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.645062663999994, 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.562 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:31:57.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:31:57.564 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.565 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:31:57.566 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.645030293999994, 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.568 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:31:57.569 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.570 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.571 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.644737917, 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.573 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:31:57.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:31:57.576 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.577 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.638913779000006, 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.578 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.579 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:31:57.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.581 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.582 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.638872276000001, 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.583 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.585 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.586 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:31:57.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.596134008999996, 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:31:57.588 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:57.589 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.590 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:31:57.591 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.592 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.593 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.596031667999995, 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:31:57.594 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.595 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:31:57.596 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:31:57.598 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:57.599 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 18.615717766, 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:57.600 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.601 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:57.603 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:57.604 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:57.606 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:31:57.606 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:57.603+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:57.707 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:57.704+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:57.754 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:57.754 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:57.754 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:31:57.754 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:57.763 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:31:57.808 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:57.805+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:57.864 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:31:57.908 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:57.906+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:57.916 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:57.916 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:31:57.916 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:31:57.916 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:31:57.916 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:57.926 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:31:58.010 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:58.007+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.693556407000003, 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.078 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.079 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:31:58.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.091305355000003, 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.082 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:31:58.083 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.084 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:31:58.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.087 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.990672717999999, 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.088 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:31:58.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:31:58.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:31:58.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 35.838145072000003, 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.095 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:31:58.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:31:58.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 35.838007832999999, 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.101 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:31:58.102 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.103 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:31:58.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 35.832850673000003, 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:31:58.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22907 3.0 3:07f1f9a0:::rbd_data.11524e94aa61.0000000000000007:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 27.097796687999999, 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053158520000000003, 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22907 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.113 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975829+0000", 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975838+0000", 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3260000000000002e-06 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975859+0000", 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0378999999999999e-05 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975870+0000", 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1488000000000001e-05 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975948+0000", 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199000000000001e-05 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976110+0000", 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000161977 2024-09-17T23:31:58.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976234+0000", 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012316599999999999 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.977277+0000", 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010435679999999999 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981028+0000", 2024-09-17T23:31:58.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037512130000000002 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981092+0000", 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3468999999999998e-05 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981143+0000", 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1017999999999998e-05 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22969 3.17 3:e879c454:::rbd_header.1152324fa224:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.184627247000002, 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058282270000000001, 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22969 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.119 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888998+0000", 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889005+0000", 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000004e-06 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889017+0000", 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2478e-05 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889029+0000", 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.168e-05 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889311+0000", 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028192400000000002 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889435+0000", 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000124046 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889886+0000", 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00045141000000000002 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.892147+0000", 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002260813 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894710+0000", 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025627710000000001 2024-09-17T23:31:58.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894768+0000", 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8059000000000001e-05 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894825+0000", 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6986e-05 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23066 3.1f 3:fea969d6:::rbd_data.1152324fa224.0000000000000007:head [delete] snapc 24=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 26.100991230000002, 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048693310000000002, 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23066 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972633+0000", 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.569999999999999e-07 2024-09-17T23:31:58.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972638+0000", 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8040000000000001e-06 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972649+0000", 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0882e-05 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.972656+0000", 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9140000000000002e-06 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973702+0000", 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010456809999999999 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973830+0000", 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128071 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.973960+0000", 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013065799999999999 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.975260+0000", 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0012997729999999999 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977401+0000", 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021405030000000002 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977457+0000", 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6168999999999997e-05 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.977502+0000", 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4919000000000001e-05 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.130 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23135 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'391, mlcod=0'0)", 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.970415380999999, 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053043279999999996, 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103209+0000", 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103215+0000", 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999998e-06 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103222+0000", 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4919999999999996e-06 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103231+0000", 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6049999999999998e-06 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103277+0000", 2024-09-17T23:31:58.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6641999999999998e-05 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103302+0000", 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4601e-05 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108442+0000", 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051404129999999999 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108513+0000", 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0192999999999997e-05 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23232 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'398, mlcod=23'1)", 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.841878490999999, 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048635629999999996, 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231745+0000", 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231746+0000", 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999996e-07 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231752+0000", 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7790000000000001e-06 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231760+0000", 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6279999999999994e-06 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231765+0000", 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8919999999999999e-06 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231809+0000", 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3544000000000002e-05 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.231837+0000", 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7815999999999999e-05 2024-09-17T23:31:58.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236575+0000", 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0047384030000000004 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.236609+0000", 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3896999999999997e-05 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24442 3.11 e145/138 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:head v 145'346, mlcod=140'344)", 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 8.9979711089999999, 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054889680000000003, 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7499999999999999e-07 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075660+0000", 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3430000000000001e-06 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075673+0000", 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2758000000000001e-05 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075680+0000", 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0190000000000001e-06 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075731+0000", 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1872000000000001e-05 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075755+0000", 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3668000000000001e-05 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081093+0000", 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053383019999999996 2024-09-17T23:31:58.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081142+0000", 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8231e-05 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24961 3.1a e150/139 3:5d9a7141:::rbd_data.1152ea45e50a.0000000000000015:head v 150'858, mlcod=146'854)", 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.8882388900000002, 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055120910000000002, 2024-09-17T23:31:58.144 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4499999999999997e-07 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185391+0000", 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3820000000000003e-06 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185401+0000", 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0271999999999999e-05 2024-09-17T23:31:58.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185408+0000", 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8859999999999998e-06 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185462+0000", 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4234000000000003e-05 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185483+0000", 2024-09-17T23:31:58.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0747e-05 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.190854+0000", 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053711899999999996 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.190897+0000", 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.148 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25112 3.17 3:e827691a:::rbd_data.1152ce7c866.0000000000000001:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.122395029, 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069415060000000001, 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25112 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951232+0000", 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1700000000000001e-06 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951250+0000", 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7629999999999999e-05 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951257+0000", 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.401e-06 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952436+0000", 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011780849999999999 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952728+0000", 2024-09-17T23:31:58.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000292379 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952856+0000", 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012781000000000001 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.954326+0000", 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014700410000000001 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958040+0000", 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037147080000000002 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958100+0000", 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9796000000000001e-05 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958170+0000", 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9950000000000006e-05 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25113 3.17 3:ea59ff92:::rbd_data.1152ce7c866.0000000000000002:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.1222887189999999, 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069436059999999997, 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.155 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25113 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951336+0000", 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2099999999999996e-07 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951339+0000", 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3560000000000001e-06 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951351+0000", 2024-09-17T23:31:58.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2422999999999999e-05 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951359+0000", 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.131e-06 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953211+0000", 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018522359999999999 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953338+0000", 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012692899999999999 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953425+0000", 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7439000000000001e-05 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955541+0000", 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021159479999999999 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958181+0000", 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026395770000000002 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958233+0000", 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2537999999999997e-05 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958279+0000", 2024-09-17T23:31:58.160 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5408000000000003e-05 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25115 3.1e e153/138 3:7ebe4efd:::rbd_data.1152ce7c866.0000000000000004:head v 153'526, mlcod=146'524)", 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.1214570479999999, 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0059104259999999999, 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.161 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:31:58.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6100000000000002e-07 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952171+0000", 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8770000000000003e-06 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952178+0000", 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6579999999999996e-06 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952183+0000", 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5180000000000004e-06 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952263+0000", 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0242999999999997e-05 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952283+0000", 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9678999999999999e-05 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958014+0000", 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057307460000000001 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958077+0000", 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3343999999999995e-05 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25119 3.11 e153/138 3:8916a254:::rbd_data.1152ce7c866.0000000000000032:head v 153'351, mlcod=146'348)", 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.121427741, 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058297339999999996, 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1199999999999999e-07 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952204+0000", 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0290000000000005e-06 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952211+0000", 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7599999999999997e-06 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952216+0000", 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0069999999999997e-06 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952606+0000", 2024-09-17T23:31:58.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038973799999999999 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952625+0000", 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9636999999999999e-05 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.957982+0000", 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053570220000000003 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958026+0000", 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3229000000000002e-05 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25124 3.e e153/152 3:71849f62:::rbd_data.1152ce7c866.0000000000000076:head v 153'706, mlcod=153'704)", 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.1177729460000001, 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051704760000000002, 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.170 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9999999999999998e-07 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955861+0000", 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955870+0000", 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8419999999999994e-06 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955874+0000", 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5410000000000002e-06 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955923+0000", 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9128000000000001e-05 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955946+0000", 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3039000000000001e-05 2024-09-17T23:31:58.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.960963+0000", 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050171520000000004 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961021+0000", 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7805e-05 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25125 3.10 e153/152 3:0a1561c8:::rbd_data.1152ce7c866.000000000000007c:head v 153'887, mlcod=0'0)", 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.117651975, 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.005055319, 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999998e-07 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955976+0000", 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7450000000000001e-06 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955982+0000", 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.694e-06 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955992+0000", 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.078e-05 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.956037+0000", 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4280000000000003e-05 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.956055+0000", 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8357999999999999e-05 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.960970+0000", 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049147610000000001 2024-09-17T23:31:58.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961027+0000", 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7258e-05 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25134 3.17 3:e95af721:::rbd_data.1152ce7c866.000000000000012d:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.961311+0000", 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 1.112312757, 2024-09-17T23:31:58.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049165739999999999, 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25134 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961311+0000", 2024-09-17T23:31:58.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961311+0000", 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961312+0000", 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1900000000000001e-07 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961315+0000", 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.523e-06 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961326+0000", 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0507e-05 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961331+0000", 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1780000000000002e-06 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961381+0000", 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9954999999999999e-05 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961567+0000", 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000186026 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961681+0000", 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000113621 2024-09-17T23:31:58.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.962459+0000", 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00077861 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.966171+0000", 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003712042 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:31:58.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.966203+0000", 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1285999999999998e-05 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.966228+0000", 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4907000000000001e-05 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:31:58.185 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:31:58.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:31:58.187 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:58.184+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:58.288 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:58.285+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:58.300 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:58.301 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:58.301 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:58.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:31:58.393 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:58.391+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:31:58.458 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:31:58.458 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:31:58.458 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:58.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:31:58.490 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:58.488+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:58.591 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:58.588+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:58.616 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.772041755999993, 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:31:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:31:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.762469656999997, 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:31:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:31:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.758939151000007, 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:31:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.757444445000004, 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:31:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.757261525999994, 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:31:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.757148025000006, 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:31:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:31:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.757123918000005, 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:31:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:31:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.757049586999997, 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:31:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:31:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.757026038000006, 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:31:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:31:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.753854837000006, 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:31:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.749446500999994, 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:31:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:31:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.722174394000007, 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:31:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:31:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:31:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.712992241999999, 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:31:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:31:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:31:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.712960749000004, 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:31:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:31:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.712935056999996, 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:31:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.712790505000001, 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:31:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:31:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.711222387999996, 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:31:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.711025040999999, 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:31:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:31:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:31:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.710998726, 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:31:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.710970383000003, 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:31:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:31:58.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:31:58.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:31:58.723 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:58.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:31:58.725 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:58.722+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:58.825 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:58.823+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:58.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:31:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:31:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:31:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:31:58.876 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:31:58.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:31:58.926 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:58.923+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:58.999 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:31:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:31:59.026 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:31:59.026 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:31:59.027 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:59.024+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:59.044 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:31:59.044 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:31:59.056 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.056 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.061 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:31:59.061 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:31:59.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:31:59.078 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:31:59.079 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:31:59.096 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:31:59.096 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:31:59.127 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:59.124+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:59.174 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.174 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:31:59.228 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:31:59.226+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:31:59.291 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.291 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:31:59.329 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:31:59.327+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:31:59.407 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.407 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:31:59.430 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:31:59.428+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:31:59.523 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.523 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.532 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:31:59.532 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:59.528+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:59.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:31:59.632 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:59.629+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:59.652 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.652 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:31:59.732 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:31:59.730+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:31:59.768 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.769 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:31:59.833 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:31:59.831+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:31:59.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:31:59.882 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:31:59.882 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:31:59.889 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:31:59.889 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:31:59.894 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-17T23:31:58.861958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":776191926,"num_objects":953,"num_object_clones":217,"num_object_copies":1906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":443,"num_read":309426,"num_read_kb":1323645,"num_write":19300,"num_write_kb":483135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7367,"num_bytes_recovered":260555591,"num_keys_recovered":1266,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17765,"ondisk_log_size":17765,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":263568,"kb_used_data":26120,"kb_used_omap":24,"kb_used_meta":237031,"kb_avail":377223792,"statfs":{"total":386547056640,"available":386277163008,"internally_reserved":0,"allocated":26746880,"data_stored":147432814,"data_compressed":6877751,"data_compressed_allocated":14286848,"data_compressed_original":141502464,"omap_allocated":24578,"internal_metadata":242720766},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21843635,"num_objects":24,"num_object_clones":15,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-27,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":36,"num_read":24835,"num_read_kb":72947,"num_write":1528,"num_write_kb":14487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":35238320,"num_keys_recovered":378,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.863432"},"pg_stats":[{"pgid":"3.1f","version":"153'415","reported_seq":5757,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865886+0000","last_change":"2024-09-17T23:31:55.869009+0000","last_active":"2024-09-17T23:31:57.865886+0000","last_peered":"2024-09-17T23:31:57.865886+0000","last_clean":"2024-09-17T23:31:57.865886+0000","last_became_active":"2024-09-17T23:31:55.865083+0000","last_became_peered":"2024-09-17T23:31:55.865083+0000","last_unstale":"2024-09-17T23:31:57.865886+0000","last_undegraded":"2024-09-17T23:31:57.865886+0000","last_fullsized":"2024-09-17T23:31:57.865886+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:15:28.758138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015669200000000001,"stat_sum":{"num_bytes":30495796,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":7434,"num_read_kb":66454,"num_write":455,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1e","version":"153'526","reported_seq":8897,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865861+0000","last_change":"2024-09-17T23:31:55.861129+0000","last_active":"2024-09-17T23:31:57.865861+0000","last_peered":"2024-09-17T23:31:57.865861+0000","last_clean":"2024-09-17T23:31:57.865861+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:31:57.865861+0000","last_undegraded":"2024-09-17T23:31:57.865861+0000","last_fullsized":"2024-09-17T23:31:57.865861+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:43:38.786461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054659999999999995,"stat_sum":{"num_bytes":80135901,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":9314,"num_read_kb":55080,"num_write":536,"num_write_kb":19041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1d","version":"153'346","reported_seq":5797,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867935+0000","last_change":"2024-09-17T23:31:55.864030+0000","last_active":"2024-09-17T23:31:57.867935+0000","last_peered":"2024-09-17T23:31:57.867935+0000","last_clean":"2024-09-17T23:31:57.867935+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:31:57.867935+0000","last_undegraded":"2024-09-17T23:31:57.867935+0000","last_fullsized":"2024-09-17T23:31:57.867935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:04.501303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095666700000000002,"stat_sum":{"num_bytes":16676352,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":4805,"num_read_kb":41204,"num_write":321,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1c","version":"153'431","reported_seq":9120,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867928+0000","last_change":"2024-09-17T23:31:56.866984+0000","last_active":"2024-09-17T23:31:57.867928+0000","last_peered":"2024-09-17T23:31:57.867928+0000","last_clean":"2024-09-17T23:31:57.867928+0000","last_became_active":"2024-09-17T23:31:56.864933+0000","last_became_peered":"2024-09-17T23:31:56.864933+0000","last_unstale":"2024-09-17T23:31:57.867928+0000","last_undegraded":"2024-09-17T23:31:57.867928+0000","last_fullsized":"2024-09-17T23:31:57.867928+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:09:29.672387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067949500000000001,"stat_sum":{"num_bytes":14323728,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":8449,"num_read_kb":40197,"num_write":521,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1b","version":"153'426","reported_seq":19545,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865567+0000","last_change":"2024-09-17T23:31:56.867268+0000","last_active":"2024-09-17T23:31:57.865567+0000","last_peered":"2024-09-17T23:31:57.865567+0000","last_clean":"2024-09-17T23:31:57.865567+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:31:57.865567+0000","last_undegraded":"2024-09-17T23:31:57.865567+0000","last_fullsized":"2024-09-17T23:31:57.865567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:28.278146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233820000000001,"stat_sum":{"num_bytes":54615313,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":18892,"num_read_kb":58432,"num_write":469,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1a","version":"154'860","reported_seq":7432,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.915325+0000","last_change":"2024-09-17T23:31:55.861734+0000","last_active":"2024-09-17T23:31:57.915325+0000","last_peered":"2024-09-17T23:31:57.915325+0000","last_clean":"2024-09-17T23:31:57.915325+0000","last_became_active":"2024-09-17T23:31:43.945744+0000","last_became_peered":"2024-09-17T23:31:43.945744+0000","last_unstale":"2024-09-17T23:31:57.915325+0000","last_undegraded":"2024-09-17T23:31:57.915325+0000","last_fullsized":"2024-09-17T23:31:57.915325+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:23.424578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048453400000000001,"stat_sum":{"num_bytes":22158013,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":11940,"num_read_kb":30844,"num_write":1193,"num_write_kb":6062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.19","version":"151'804","reported_seq":6525,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.893584+0000","last_change":"2024-09-17T23:31:57.869359+0000","last_active":"2024-09-17T23:31:57.893584+0000","last_peered":"2024-09-17T23:31:57.893584+0000","last_clean":"2024-09-17T23:31:57.893584+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:31:57.893584+0000","last_undegraded":"2024-09-17T23:31:57.893584+0000","last_fullsized":"2024-09-17T23:31:57.893584+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:37.078828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010618450000000001,"stat_sum":{"num_bytes":15059999,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":8535,"num_read_kb":17540,"num_write":912,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.18","version":"153'436","reported_seq":4273,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865309+0000","last_change":"2024-09-17T23:31:55.860542+0000","last_active":"2024-09-17T23:31:57.865309+0000","last_peered":"2024-09-17T23:31:57.865309+0000","last_clean":"2024-09-17T23:31:57.865309+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:31:57.865309+0000","last_undegraded":"2024-09-17T23:31:57.865309+0000","last_fullsized":"2024-09-17T23:31:57.865309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:15.051794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011217,"stat_sum":{"num_bytes":17261093,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":4989,"num_read_kb":22192,"num_write":434,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.17","version":"147'621","reported_seq":8014,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-17T23:31:55.003221+0000","last_change":"2024-09-17T23:31:52.002603+0000","last_active":"2024-09-17T23:31:55.003221+0000","last_peered":"2024-09-17T23:31:55.003221+0000","last_clean":"2024-09-17T23:31:55.003221+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:31:55.003221+0000","last_undegraded":"2024-09-17T23:31:55.003221+0000","last_fullsized":"2024-09-17T23:31:55.003221+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:49:24.049727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010706089999999999,"stat_sum":{"num_bytes":36254810,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":10862,"num_read_kb":28901,"num_write":531,"num_write_kb":13269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"18","length":"a"}]},{"pgid":"3.16","version":"146'351","reported_seq":4292,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865788+0000","last_change":"2024-09-17T23:31:55.869727+0000","last_active":"2024-09-17T23:31:57.865788+0000","last_peered":"2024-09-17T23:31:57.865788+0000","last_clean":"2024-09-17T23:31:57.865788+0000","last_became_active":"2024-09-17T23:31:55.866071+0000","last_became_peered":"2024-09-17T23:31:55.866071+0000","last_unstale":"2024-09-17T23:31:57.865788+0000","last_undegraded":"2024-09-17T23:31:57.865788+0000","last_fullsized":"2024-09-17T23:31:57.865788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:51:17.970885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557527,"stat_sum":{"num_bytes":14573104,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4431,"num_read_kb":31183,"num_write":358,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.15","version":"146'581","reported_seq":12764,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868103+0000","last_change":"2024-09-17T23:31:55.865584+0000","last_active":"2024-09-17T23:31:57.868103+0000","last_peered":"2024-09-17T23:31:57.868103+0000","last_clean":"2024-09-17T23:31:57.868103+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:31:57.868103+0000","last_undegraded":"2024-09-17T23:31:57.868103+0000","last_fullsized":"2024-09-17T23:31:57.868103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:59.053768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021055800000000001,"stat_sum":{"num_bytes":27632680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":14519,"num_read_kb":31572,"num_write":615,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.14","version":"154'542","reported_seq":7889,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.917432+0000","last_change":"2024-09-17T23:31:55.865619+0000","last_active":"2024-09-17T23:31:57.917432+0000","last_peered":"2024-09-17T23:31:57.917432+0000","last_clean":"2024-09-17T23:31:57.917432+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:31:57.917432+0000","last_undegraded":"2024-09-17T23:31:57.917432+0000","last_fullsized":"2024-09-17T23:31:57.917432+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:11.852436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573870000000001,"stat_sum":{"num_bytes":12967009,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":8471,"num_read_kb":33614,"num_write":457,"num_write_kb":18367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":243,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868175+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:31:57.868175+0000","last_peered":"2024-09-17T23:31:57.868175+0000","last_clean":"2024-09-17T23:31:57.868175+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:31:57.868175+0000","last_undegraded":"2024-09-17T23:31:57.868175+0000","last_fullsized":"2024-09-17T23:31:57.868175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"154'606","reported_seq":5036,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.864016+0000","last_change":"2024-09-17T23:31:55.862884+0000","last_active":"2024-09-17T23:31:57.864016+0000","last_peered":"2024-09-17T23:31:57.864016+0000","last_clean":"2024-09-17T23:31:57.864016+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:31:57.864016+0000","last_undegraded":"2024-09-17T23:31:57.864016+0000","last_fullsized":"2024-09-17T23:31:57.864016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:29.368819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057116299999999997,"stat_sum":{"num_bytes":5341271,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":5974,"num_read_kb":22618,"num_write":873,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863742+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:31:57.863742+0000","last_peered":"2024-09-17T23:31:57.863742+0000","last_clean":"2024-09-17T23:31:57.863742+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:31:57.863742+0000","last_undegraded":"2024-09-17T23:31:57.863742+0000","last_fullsized":"2024-09-17T23:31:57.863742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"153'415","reported_seq":12464,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863880+0000","last_change":"2024-09-17T23:31:55.863199+0000","last_active":"2024-09-17T23:31:57.863880+0000","last_peered":"2024-09-17T23:31:57.863880+0000","last_clean":"2024-09-17T23:31:57.863880+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:31:57.863880+0000","last_undegraded":"2024-09-17T23:31:57.863880+0000","last_fullsized":"2024-09-17T23:31:57.863880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:03.965375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061819999999999996,"stat_sum":{"num_bytes":15466539,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":13024,"num_read_kb":60475,"num_write":418,"num_write_kb":30569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.5","version":"0'0","reported_seq":253,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863625+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:31:57.863625+0000","last_peered":"2024-09-17T23:31:57.863625+0000","last_clean":"2024-09-17T23:31:57.863625+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:31:57.863625+0000","last_undegraded":"2024-09-17T23:31:57.863625+0000","last_fullsized":"2024-09-17T23:31:57.863625+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"150'383","reported_seq":7223,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863651+0000","last_change":"2024-09-17T23:31:55.862882+0000","last_active":"2024-09-17T23:31:57.863651+0000","last_peered":"2024-09-17T23:31:57.863651+0000","last_clean":"2024-09-17T23:31:57.863651+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:31:57.863651+0000","last_undegraded":"2024-09-17T23:31:57.863651+0000","last_fullsized":"2024-09-17T23:31:57.863651+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:57.672171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050142700000000004,"stat_sum":{"num_bytes":9613342,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":4593,"num_read_kb":19709,"num_write":434,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865381+0000","last_change":"2024-09-17T23:31:54.009886+0000","last_active":"2024-09-17T23:31:57.865381+0000","last_peered":"2024-09-17T23:31:57.865381+0000","last_clean":"2024-09-17T23:31:57.865381+0000","last_became_active":"2024-09-17T23:31:54.009591+0000","last_became_peered":"2024-09-17T23:31:54.009591+0000","last_unstale":"2024-09-17T23:31:57.865381+0000","last_undegraded":"2024-09-17T23:31:57.865381+0000","last_fullsized":"2024-09-17T23:31:57.865381+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:40.324116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"150'371","reported_seq":10470,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865704+0000","last_change":"2024-09-17T23:31:55.859972+0000","last_active":"2024-09-17T23:31:57.865704+0000","last_peered":"2024-09-17T23:31:57.865704+0000","last_clean":"2024-09-17T23:31:57.865704+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:31:57.865704+0000","last_undegraded":"2024-09-17T23:31:57.865704+0000","last_fullsized":"2024-09-17T23:31:57.865704+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:29.022075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053232300000000002,"stat_sum":{"num_bytes":17972768,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":10196,"num_read_kb":27182,"num_write":429,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.3","version":"154'570","reported_seq":7311,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.889915+0000","last_change":"2024-09-17T23:31:55.866386+0000","last_active":"2024-09-17T23:31:57.889915+0000","last_peered":"2024-09-17T23:31:57.889915+0000","last_clean":"2024-09-17T23:31:57.889915+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:31:57.889915+0000","last_undegraded":"2024-09-17T23:31:57.889915+0000","last_fullsized":"2024-09-17T23:31:57.889915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:40.742854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018497380000000001,"stat_sum":{"num_bytes":17055232,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":24,"num_read":7271,"num_read_kb":29731,"num_write":591,"num_write_kb":9153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.2","version":"18'2","reported_seq":346,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867985+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:31:57.867985+0000","last_peered":"2024-09-17T23:31:57.867985+0000","last_clean":"2024-09-17T23:31:57.867985+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:31:57.867985+0000","last_undegraded":"2024-09-17T23:31:57.867985+0000","last_fullsized":"2024-09-17T23:31:57.867985+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"151'646","reported_seq":12000,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863325+0000","last_change":"2024-09-17T23:31:56.867801+0000","last_active":"2024-09-17T23:31:57.863325+0000","last_peered":"2024-09-17T23:31:57.863325+0000","last_clean":"2024-09-17T23:31:57.863325+0000","last_became_active":"2024-09-17T23:31:56.865272+0000","last_became_peered":"2024-09-17T23:31:56.865272+0000","last_unstale":"2024-09-17T23:31:57.863325+0000","last_undegraded":"2024-09-17T23:31:57.863325+0000","last_fullsized":"2024-09-17T23:31:57.863325+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:34.785652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000816283,"stat_sum":{"num_bytes":10143843,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":12767,"num_read_kb":56381,"num_write":711,"num_write_kb":10223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":297,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865836+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:31:57.865836+0000","last_peered":"2024-09-17T23:31:57.865836+0000","last_clean":"2024-09-17T23:31:57.865836+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:31:57.865836+0000","last_undegraded":"2024-09-17T23:31:57.865836+0000","last_fullsized":"2024-09-17T23:31:57.865836+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"151'857","reported_seq":8998,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863387+0000","last_change":"2024-09-17T23:31:56.868388+0000","last_active":"2024-09-17T23:31:57.863387+0000","last_peered":"2024-09-17T23:31:57.863387+0000","last_clean":"2024-09-17T23:31:57.863387+0000","last_became_active":"2024-09-17T23:31:56.865940+0000","last_became_peered":"2024-09-17T23:31:56.865940+0000","last_unstale":"2024-09-17T23:31:57.863387+0000","last_undegraded":"2024-09-17T23:31:57.863387+0000","last_fullsized":"2024-09-17T23:31:57.863387+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:25:08.005325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062989100000000004,"stat_sum":{"num_bytes":26297531,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11450,"num_read_kb":16034,"num_write":683,"num_write_kb":4230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863340+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:31:57.863340+0000","last_peered":"2024-09-17T23:31:57.863340+0000","last_clean":"2024-09-17T23:31:57.863340+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:31:57.863340+0000","last_undegraded":"2024-09-17T23:31:57.863340+0000","last_fullsized":"2024-09-17T23:31:57.863340+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"147'670","reported_seq":11205,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865337+0000","last_change":"2024-09-17T23:31:55.871630+0000","last_active":"2024-09-17T23:31:57.865337+0000","last_peered":"2024-09-17T23:31:57.865337+0000","last_clean":"2024-09-17T23:31:57.865337+0000","last_became_active":"2024-09-17T23:31:55.869290+0000","last_became_peered":"2024-09-17T23:31:55.869290+0000","last_unstale":"2024-09-17T23:31:57.865337+0000","last_undegraded":"2024-09-17T23:31:57.865337+0000","last_fullsized":"2024-09-17T23:31:57.865337+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:56:17.415710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086085100000000004,"stat_sum":{"num_bytes":23587937,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11181,"num_read_kb":79280,"num_write":755,"num_write_kb":59941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.3","version":"16'1","reported_seq":215,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863712+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:31:57.863712+0000","last_peered":"2024-09-17T23:31:57.863712+0000","last_clean":"2024-09-17T23:31:57.863712+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:31:57.863712+0000","last_undegraded":"2024-09-17T23:31:57.863712+0000","last_fullsized":"2024-09-17T23:31:57.863712+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":381,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863392+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:31:57.863392+0000","last_peered":"2024-09-17T23:31:57.863392+0000","last_clean":"2024-09-17T23:31:57.863392+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:31:57.863392+0000","last_undegraded":"2024-09-17T23:31:57.863392+0000","last_fullsized":"2024-09-17T23:31:57.863392+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"150'432","reported_seq":14666,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863475+0000","last_change":"2024-09-17T23:31:55.862141+0000","last_active":"2024-09-17T23:31:57.863475+0000","last_peered":"2024-09-17T23:31:57.863475+0000","last_clean":"2024-09-17T23:31:57.863475+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:31:57.863475+0000","last_undegraded":"2024-09-17T23:31:57.863475+0000","last_fullsized":"2024-09-17T23:31:57.863475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:03:52.265763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044238099999999999,"stat_sum":{"num_bytes":22973693,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":15144,"num_read_kb":73922,"num_write":383,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.9","version":"153'697","reported_seq":11456,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867824+0000","last_change":"2024-09-17T23:31:55.869675+0000","last_active":"2024-09-17T23:31:57.867824+0000","last_peered":"2024-09-17T23:31:57.867824+0000","last_clean":"2024-09-17T23:31:57.867824+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:31:57.867824+0000","last_undegraded":"2024-09-17T23:31:57.867824+0000","last_fullsized":"2024-09-17T23:31:57.867824+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:31.783060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087999400000000005,"stat_sum":{"num_bytes":12578853,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":11139,"num_read_kb":39046,"num_write":813,"num_write_kb":25901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.a","version":"154'596","reported_seq":9750,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.882025+0000","last_change":"2024-09-17T23:31:55.866693+0000","last_active":"2024-09-17T23:31:57.882025+0000","last_peered":"2024-09-17T23:31:57.882025+0000","last_clean":"2024-09-17T23:31:57.882025+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:31:57.882025+0000","last_undegraded":"2024-09-17T23:31:57.882025+0000","last_fullsized":"2024-09-17T23:31:57.882025+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:48.541901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015858700000000001,"stat_sum":{"num_bytes":12512784,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":10955,"num_read_kb":53295,"num_write":595,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.b","version":"154'914","reported_seq":11472,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.872547+0000","last_change":"2024-09-17T23:31:57.872211+0000","last_active":"2024-09-17T23:31:57.872547+0000","last_peered":"2024-09-17T23:31:57.872547+0000","last_clean":"2024-09-17T23:31:57.872547+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:31:57.872547+0000","last_undegraded":"2024-09-17T23:31:57.872547+0000","last_fullsized":"2024-09-17T23:31:57.872547+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:11.321721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080631599999999996,"stat_sum":{"num_bytes":22895136,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":15013,"num_read_kb":61473,"num_write":890,"num_write_kb":11836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.c","version":"154'458","reported_seq":5329,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.918450+0000","last_change":"2024-09-17T23:31:55.865938+0000","last_active":"2024-09-17T23:31:57.918450+0000","last_peered":"2024-09-17T23:31:57.918450+0000","last_clean":"2024-09-17T23:31:57.918450+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:31:57.918450+0000","last_undegraded":"2024-09-17T23:31:57.918450+0000","last_fullsized":"2024-09-17T23:31:57.918450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:11.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019546559999999999,"stat_sum":{"num_bytes":9347621,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":14,"num_read":5960,"num_read_kb":36791,"num_write":451,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.d","version":"153'408","reported_seq":9657,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867994+0000","last_change":"2024-09-17T23:31:55.864311+0000","last_active":"2024-09-17T23:31:57.867994+0000","last_peered":"2024-09-17T23:31:57.867994+0000","last_clean":"2024-09-17T23:31:57.867994+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:31:57.867994+0000","last_undegraded":"2024-09-17T23:31:57.867994+0000","last_fullsized":"2024-09-17T23:31:57.867994+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:05:43.654480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028153,"stat_sum":{"num_bytes":26217540,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":9068,"num_read_kb":42813,"num_write":455,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.e","version":"154'710","reported_seq":10052,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.885815+0000","last_change":"2024-09-17T23:31:56.867944+0000","last_active":"2024-09-17T23:31:57.885815+0000","last_peered":"2024-09-17T23:31:57.885815+0000","last_clean":"2024-09-17T23:31:57.885815+0000","last_became_active":"2024-09-17T23:31:56.865052+0000","last_became_peered":"2024-09-17T23:31:56.865052+0000","last_unstale":"2024-09-17T23:31:57.885815+0000","last_undegraded":"2024-09-17T23:31:57.885815+0000","last_fullsized":"2024-09-17T23:31:57.885815+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:10.451905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066576600000000004,"stat_sum":{"num_bytes":13903445,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":10354,"num_read_kb":26378,"num_write":771,"num_write_kb":7195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.f","version":"150'358","reported_seq":4749,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868100+0000","last_change":"2024-09-17T23:31:55.871364+0000","last_active":"2024-09-17T23:31:57.868100+0000","last_peered":"2024-09-17T23:31:57.868100+0000","last_clean":"2024-09-17T23:31:57.868100+0000","last_became_active":"2024-09-17T23:31:55.869346+0000","last_became_peered":"2024-09-17T23:31:55.869346+0000","last_unstale":"2024-09-17T23:31:57.868100+0000","last_undegraded":"2024-09-17T23:31:57.868100+0000","last_fullsized":"2024-09-17T23:31:57.868100+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:05:21.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088462000000000002,"stat_sum":{"num_bytes":6846004,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":4817,"num_read_kb":27404,"num_write":373,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.10","version":"153'888","reported_seq":7793,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867815+0000","last_change":"2024-09-17T23:31:56.868103+0000","last_active":"2024-09-17T23:31:57.867815+0000","last_peered":"2024-09-17T23:31:57.867815+0000","last_clean":"2024-09-17T23:31:57.867815+0000","last_became_active":"2024-09-17T23:31:56.865636+0000","last_became_peered":"2024-09-17T23:31:56.865636+0000","last_unstale":"2024-09-17T23:31:57.867815+0000","last_undegraded":"2024-09-17T23:31:57.867815+0000","last_fullsized":"2024-09-17T23:31:57.867815+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:27.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060585100000000002,"stat_sum":{"num_bytes":21892689,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":9609,"num_read_kb":34061,"num_write":1199,"num_write_kb":24063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.11","version":"153'351","reported_seq":8487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865638+0000","last_change":"2024-09-17T23:31:55.861212+0000","last_active":"2024-09-17T23:31:57.865638+0000","last_peered":"2024-09-17T23:31:57.865638+0000","last_clean":"2024-09-17T23:31:57.865638+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:31:57.865638+0000","last_undegraded":"2024-09-17T23:31:57.865638+0000","last_fullsized":"2024-09-17T23:31:57.865638+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:18.805689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002261079,"stat_sum":{"num_bytes":50728448,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":8068,"num_read_kb":46907,"num_write":329,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.12","version":"153'427","reported_seq":5741,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867745+0000","last_change":"2024-09-17T23:31:55.866468+0000","last_active":"2024-09-17T23:31:57.867745+0000","last_peered":"2024-09-17T23:31:57.867745+0000","last_clean":"2024-09-17T23:31:57.867745+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:31:57.867745+0000","last_undegraded":"2024-09-17T23:31:57.867745+0000","last_fullsized":"2024-09-17T23:31:57.867745+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:30:58.025203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001367851,"stat_sum":{"num_bytes":61967360,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":5018,"num_read_kb":38008,"num_write":480,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.13","version":"153'549","reported_seq":15105,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868645+0000","last_change":"2024-09-17T23:31:55.866024+0000","last_active":"2024-09-17T23:31:57.868645+0000","last_peered":"2024-09-17T23:31:57.868645+0000","last_clean":"2024-09-17T23:31:57.868645+0000","last_became_active":"2024-09-17T23:31:43.947677+0000","last_became_peered":"2024-09-17T23:31:43.947677+0000","last_unstale":"2024-09-17T23:31:57.868645+0000","last_undegraded":"2024-09-17T23:31:57.868645+0000","last_fullsized":"2024-09-17T23:31:57.868645+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:38:06.492403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017458930000000001,"stat_sum":{"num_bytes":46105705,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":15068,"num_read_kb":74824,"num_write":704,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":775601539,"num_objects":949,"num_object_clones":217,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":443,"num_read":309310,"num_read_kb":1323545,"num_write":19139,"num_write_kb":480717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7354,"num_bytes_recovered":259965128,"num_keys_recovered":1266,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24731648,"data_stored":178905081,"data_compressed":8916499,"data_compressed_allocated":18247680,"data_compressed_original":175307264,"omap_allocated":7598,"internal_metadata":0},"log_size":17645,"ondisk_log_size":17645,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500682,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":7016,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":7184384,"data_stored":38351943,"data_compressed":1894198,"data_compressed_allocated":3923968,"data_compressed_original":36782592,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640295,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50548,"kb_used_data":5464,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321292,"statfs":{"total":96636764160,"available":96585003008,"internally_reserved":0,"allocated":5595136,"data_stored":27751710,"data_compressed":1305723,"data_compressed_allocated":2613248,"data_compressed_original":26359296,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640295,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78364,"kb_used_data":6644,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293476,"statfs":{"total":96636764160,"available":96556519424,"internally_reserved":0,"allocated":6803456,"data_stored":41139664,"data_compressed":1709488,"data_compressed_allocated":3661824,"data_compressed_original":39647232,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640295,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640296,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86788,"kb_used_data":6996,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94285052,"statfs":{"total":96636764160,"available":96547893248,"internally_reserved":0,"allocated":7163904,"data_stored":40189497,"data_compressed":1968342,"data_compressed_allocated":4087808,"data_compressed_original":38713344,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5308416,"data_stored":39361289,"data_compressed":1968306,"data_compressed_allocated":4083712,"data_compressed_original":38660096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":2250800,"data_compressed":129518,"data_compressed_allocated":262144,"data_compressed_original":2220032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4956160,"data_stored":40311594,"data_compressed":1709452,"data_compressed_allocated":3657728,"data_compressed_original":39593984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3706880,"data_stored":26386228,"data_compressed":1299945,"data_compressed_allocated":2576384,"data_compressed_original":25769472,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5136384,"data_stored":36410070,"data_compressed":1823159,"data_compressed_allocated":3772416,"data_compressed_original":35656192,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:31:59.896 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T23:31:59.896 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:31:59.896 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T23:31:59.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:31:59.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:31:59.935 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:31:59.932+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:00.005 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.005 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:00.060 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:00.057+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:00.130 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.130 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.136 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:00.134+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:00.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:00.238 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:00.235+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:00.250 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:00.250 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 156, 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:32:00.027019+0000", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:31:11.038388+0000", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:31:38.858088+0000", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5804032, 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:32:00.251 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:32:00.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:32:00.253 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:32:00.254 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:32:00.255 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:32:00.256 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:32:00.257 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:32:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:32:00.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:32:00.260 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:32:00.261 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:32:00.262 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:32:00.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:32:00.264 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.9900000095367432, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.9900000095367432, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:32:00.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:32:00.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "156", 2024-09-17T23:32:00.267 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 45, 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 156, 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:32:00.268 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.269 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:32:00.270 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.6099998950958252, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.6099998950958252, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 152, 2024-09-17T23:32:00.272 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:32:00.273 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:32:00.274 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:32:00.276 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.277 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 138, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:00.279 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:00.280 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:00.281 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:32:00.282 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:32:00.283 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:32:00.284 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:00.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:32:00.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 153, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:00.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.295 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 153, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:00.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.298 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.1843414306640625, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 11, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 93, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 107, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 150, 2024-09-17T23:32:00.302 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 94, 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:32:00.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:00.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1359318569", 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1359318569", 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1359318569", 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1359318569", 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.307 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.308 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 55177, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:00.309 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:32:00.310 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.311 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 94 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.12", 2024-09-17T23:32:00.312 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: 2, 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:32:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 21, 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 13 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 21, 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:32:00.316 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:00.317 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:00.319 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.319 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.319 INFO:tasks.thrashosds.thrasher:j is {'epoch': 156, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:32:00.027019+0000', 'last_up_change': '2024-09-17T23:31:11.038388+0000', 'last_in_change': '2024-09-17T23:31:38.858088+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 156, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.609999895095825, 'score_stable': 2.609999895095825, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 152, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1843414306640625, 'last_clean_begin': 11, 'last_clean_end': 93, 'up_from': 107, 'up_thru': 150, 'down_at': 94, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1359318569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1359318569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1359318569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1359318569}]}, 'public_addr': '172.21.15.37:6809/1359318569', 'cluster_addr': '172.21.15.37:6811/1359318569', 'heartbeat_back_addr': '172.21.15.37:6815/1359318569', 'heartbeat_front_addr': '172.21.15.37:6813/1359318569', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 55177, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 94}], 'pg_upmap': [{'pgid': '3.12', 'osds': [2, 1]}, {'pgid': '3.17', 'osds': [5, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 13}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:32:00.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:00.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:00.360 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:00.357+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:00.440 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:00.437+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:00.467 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.467 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:00.560 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:00.557+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:00.613 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:00.614 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:00.623 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.624 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.626 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-17T23:32:00.392244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":774120374,"num_objects":948,"num_object_clones":217,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":443,"num_read":309426,"num_read_kb":1323645,"num_write":19305,"num_write_kb":483135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7367,"num_bytes_recovered":260555591,"num_keys_recovered":1266,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17770,"ondisk_log_size":17770,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":261824,"kb_used_data":24340,"kb_used_omap":24,"kb_used_meta":237031,"kb_avail":377225536,"statfs":{"total":386547056640,"available":386278948864,"internally_reserved":0,"allocated":24924160,"data_stored":139822376,"data_compressed":6190626,"data_compressed_allocated":13070336,"data_compressed_original":134226432,"omap_allocated":24578,"internal_metadata":242720766},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23653043,"num_objects":22,"num_object_clones":16,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-27,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":37,"num_read":24667,"num_read_kb":72801,"num_write":1516,"num_write_kb":14481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2070,"num_bytes_recovered":35238320,"num_keys_recovered":378,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001067"},"pg_stats":[{"pgid":"3.1f","version":"153'415","reported_seq":5757,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865886+0000","last_change":"2024-09-17T23:31:55.869009+0000","last_active":"2024-09-17T23:31:57.865886+0000","last_peered":"2024-09-17T23:31:57.865886+0000","last_clean":"2024-09-17T23:31:57.865886+0000","last_became_active":"2024-09-17T23:31:55.865083+0000","last_became_peered":"2024-09-17T23:31:55.865083+0000","last_unstale":"2024-09-17T23:31:57.865886+0000","last_undegraded":"2024-09-17T23:31:57.865886+0000","last_fullsized":"2024-09-17T23:31:57.865886+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:15:28.758138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015669200000000001,"stat_sum":{"num_bytes":30495796,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":7,"num_read":7434,"num_read_kb":66454,"num_write":455,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1e","version":"153'526","reported_seq":8897,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865861+0000","last_change":"2024-09-17T23:31:55.861129+0000","last_active":"2024-09-17T23:31:57.865861+0000","last_peered":"2024-09-17T23:31:57.865861+0000","last_clean":"2024-09-17T23:31:57.865861+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:31:57.865861+0000","last_undegraded":"2024-09-17T23:31:57.865861+0000","last_fullsized":"2024-09-17T23:31:57.865861+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:43:38.786461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054659999999999995,"stat_sum":{"num_bytes":80135901,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":9314,"num_read_kb":55080,"num_write":536,"num_write_kb":19041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1d","version":"153'346","reported_seq":5797,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867935+0000","last_change":"2024-09-17T23:31:55.864030+0000","last_active":"2024-09-17T23:31:57.867935+0000","last_peered":"2024-09-17T23:31:57.867935+0000","last_clean":"2024-09-17T23:31:57.867935+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:31:57.867935+0000","last_undegraded":"2024-09-17T23:31:57.867935+0000","last_fullsized":"2024-09-17T23:31:57.867935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:23:04.501303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095666700000000002,"stat_sum":{"num_bytes":16676352,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":4805,"num_read_kb":41204,"num_write":321,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1c","version":"153'431","reported_seq":9120,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867928+0000","last_change":"2024-09-17T23:31:56.866984+0000","last_active":"2024-09-17T23:31:57.867928+0000","last_peered":"2024-09-17T23:31:57.867928+0000","last_clean":"2024-09-17T23:31:57.867928+0000","last_became_active":"2024-09-17T23:31:56.864933+0000","last_became_peered":"2024-09-17T23:31:56.864933+0000","last_unstale":"2024-09-17T23:31:57.867928+0000","last_undegraded":"2024-09-17T23:31:57.867928+0000","last_fullsized":"2024-09-17T23:31:57.867928+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:09:29.672387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067949500000000001,"stat_sum":{"num_bytes":14323728,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":12,"num_read":8449,"num_read_kb":40197,"num_write":521,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1b","version":"153'426","reported_seq":19545,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865567+0000","last_change":"2024-09-17T23:31:56.867268+0000","last_active":"2024-09-17T23:31:57.865567+0000","last_peered":"2024-09-17T23:31:57.865567+0000","last_clean":"2024-09-17T23:31:57.865567+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:31:57.865567+0000","last_undegraded":"2024-09-17T23:31:57.865567+0000","last_fullsized":"2024-09-17T23:31:57.865567+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:28.278146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012233820000000001,"stat_sum":{"num_bytes":54615313,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":22,"num_read":18892,"num_read_kb":58432,"num_write":469,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.1a","version":"154'860","reported_seq":7432,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.915325+0000","last_change":"2024-09-17T23:31:55.861734+0000","last_active":"2024-09-17T23:31:57.915325+0000","last_peered":"2024-09-17T23:31:57.915325+0000","last_clean":"2024-09-17T23:31:57.915325+0000","last_became_active":"2024-09-17T23:31:43.945744+0000","last_became_peered":"2024-09-17T23:31:43.945744+0000","last_unstale":"2024-09-17T23:31:57.915325+0000","last_undegraded":"2024-09-17T23:31:57.915325+0000","last_fullsized":"2024-09-17T23:31:57.915325+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:23.424578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048453400000000001,"stat_sum":{"num_bytes":22158013,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":11940,"num_read_kb":30844,"num_write":1193,"num_write_kb":6062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.19","version":"151'804","reported_seq":6525,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.893584+0000","last_change":"2024-09-17T23:31:57.869359+0000","last_active":"2024-09-17T23:31:57.893584+0000","last_peered":"2024-09-17T23:31:57.893584+0000","last_clean":"2024-09-17T23:31:57.893584+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:31:57.893584+0000","last_undegraded":"2024-09-17T23:31:57.893584+0000","last_fullsized":"2024-09-17T23:31:57.893584+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:37.078828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010618450000000001,"stat_sum":{"num_bytes":15059999,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":8535,"num_read_kb":17540,"num_write":912,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.18","version":"153'436","reported_seq":4273,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865309+0000","last_change":"2024-09-17T23:31:55.860542+0000","last_active":"2024-09-17T23:31:57.865309+0000","last_peered":"2024-09-17T23:31:57.865309+0000","last_clean":"2024-09-17T23:31:57.865309+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:31:57.865309+0000","last_undegraded":"2024-09-17T23:31:57.865309+0000","last_fullsized":"2024-09-17T23:31:57.865309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:15.051794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001011217,"stat_sum":{"num_bytes":17261093,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":17,"num_read":4989,"num_read_kb":22192,"num_write":434,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.17","version":"155'626","reported_seq":8044,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T23:32:00.050616+0000","last_change":"2024-09-17T23:32:00.050906+0000","last_active":"2024-09-17T23:32:00.050616+0000","last_peered":"2024-09-17T23:32:00.050616+0000","last_clean":"2024-09-17T23:32:00.050616+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:00.050616+0000","last_undegraded":"2024-09-17T23:32:00.050616+0000","last_fullsized":"2024-09-17T23:32:00.050616+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:59:58.998551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049477400000000002,"stat_sum":{"num_bytes":34183258,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":10862,"num_read_kb":28901,"num_write":536,"num_write_kb":13269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"15","length":"d"}]},{"pgid":"3.16","version":"146'351","reported_seq":4292,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865788+0000","last_change":"2024-09-17T23:31:55.869727+0000","last_active":"2024-09-17T23:31:57.865788+0000","last_peered":"2024-09-17T23:31:57.865788+0000","last_clean":"2024-09-17T23:31:57.865788+0000","last_became_active":"2024-09-17T23:31:55.866071+0000","last_became_peered":"2024-09-17T23:31:55.866071+0000","last_unstale":"2024-09-17T23:31:57.865788+0000","last_undegraded":"2024-09-17T23:31:57.865788+0000","last_fullsized":"2024-09-17T23:31:57.865788+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:51:17.970885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001557527,"stat_sum":{"num_bytes":14573104,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":4431,"num_read_kb":31183,"num_write":358,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.15","version":"146'581","reported_seq":12764,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868103+0000","last_change":"2024-09-17T23:31:55.865584+0000","last_active":"2024-09-17T23:31:57.868103+0000","last_peered":"2024-09-17T23:31:57.868103+0000","last_clean":"2024-09-17T23:31:57.868103+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:31:57.868103+0000","last_undegraded":"2024-09-17T23:31:57.868103+0000","last_fullsized":"2024-09-17T23:31:57.868103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:23:59.053768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021055800000000001,"stat_sum":{"num_bytes":27632680,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":14519,"num_read_kb":31572,"num_write":615,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.14","version":"154'542","reported_seq":7889,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.917432+0000","last_change":"2024-09-17T23:31:55.865619+0000","last_active":"2024-09-17T23:31:57.917432+0000","last_peered":"2024-09-17T23:31:57.917432+0000","last_clean":"2024-09-17T23:31:57.917432+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:31:57.917432+0000","last_undegraded":"2024-09-17T23:31:57.917432+0000","last_fullsized":"2024-09-17T23:31:57.917432+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:11.852436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018573870000000001,"stat_sum":{"num_bytes":12967009,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":8471,"num_read_kb":33614,"num_write":457,"num_write_kb":18367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":243,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868175+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:31:57.868175+0000","last_peered":"2024-09-17T23:31:57.868175+0000","last_clean":"2024-09-17T23:31:57.868175+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:31:57.868175+0000","last_undegraded":"2024-09-17T23:31:57.868175+0000","last_fullsized":"2024-09-17T23:31:57.868175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"154'606","reported_seq":5036,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.864016+0000","last_change":"2024-09-17T23:31:55.862884+0000","last_active":"2024-09-17T23:31:57.864016+0000","last_peered":"2024-09-17T23:31:57.864016+0000","last_clean":"2024-09-17T23:31:57.864016+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:31:57.864016+0000","last_undegraded":"2024-09-17T23:31:57.864016+0000","last_fullsized":"2024-09-17T23:31:57.864016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:29.368819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057116299999999997,"stat_sum":{"num_bytes":5341271,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":11,"num_read":5974,"num_read_kb":22618,"num_write":873,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863742+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:31:57.863742+0000","last_peered":"2024-09-17T23:31:57.863742+0000","last_clean":"2024-09-17T23:31:57.863742+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:31:57.863742+0000","last_undegraded":"2024-09-17T23:31:57.863742+0000","last_fullsized":"2024-09-17T23:31:57.863742+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"153'415","reported_seq":12464,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863880+0000","last_change":"2024-09-17T23:31:55.863199+0000","last_active":"2024-09-17T23:31:57.863880+0000","last_peered":"2024-09-17T23:31:57.863880+0000","last_clean":"2024-09-17T23:31:57.863880+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:31:57.863880+0000","last_undegraded":"2024-09-17T23:31:57.863880+0000","last_fullsized":"2024-09-17T23:31:57.863880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:03.965375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061819999999999996,"stat_sum":{"num_bytes":15466539,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":13024,"num_read_kb":60475,"num_write":418,"num_write_kb":30569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.5","version":"0'0","reported_seq":253,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863625+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:31:57.863625+0000","last_peered":"2024-09-17T23:31:57.863625+0000","last_clean":"2024-09-17T23:31:57.863625+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:31:57.863625+0000","last_undegraded":"2024-09-17T23:31:57.863625+0000","last_fullsized":"2024-09-17T23:31:57.863625+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"150'383","reported_seq":7223,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863651+0000","last_change":"2024-09-17T23:31:55.862882+0000","last_active":"2024-09-17T23:31:57.863651+0000","last_peered":"2024-09-17T23:31:57.863651+0000","last_clean":"2024-09-17T23:31:57.863651+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:31:57.863651+0000","last_undegraded":"2024-09-17T23:31:57.863651+0000","last_fullsized":"2024-09-17T23:31:57.863651+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:57.672171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050142700000000004,"stat_sum":{"num_bytes":9613342,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":4593,"num_read_kb":19709,"num_write":434,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865381+0000","last_change":"2024-09-17T23:31:54.009886+0000","last_active":"2024-09-17T23:31:57.865381+0000","last_peered":"2024-09-17T23:31:57.865381+0000","last_clean":"2024-09-17T23:31:57.865381+0000","last_became_active":"2024-09-17T23:31:54.009591+0000","last_became_peered":"2024-09-17T23:31:54.009591+0000","last_unstale":"2024-09-17T23:31:57.865381+0000","last_undegraded":"2024-09-17T23:31:57.865381+0000","last_fullsized":"2024-09-17T23:31:57.865381+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:40.324116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"150'371","reported_seq":10470,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865704+0000","last_change":"2024-09-17T23:31:55.859972+0000","last_active":"2024-09-17T23:31:57.865704+0000","last_peered":"2024-09-17T23:31:57.865704+0000","last_clean":"2024-09-17T23:31:57.865704+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:31:57.865704+0000","last_undegraded":"2024-09-17T23:31:57.865704+0000","last_fullsized":"2024-09-17T23:31:57.865704+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:29.022075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053232300000000002,"stat_sum":{"num_bytes":17972768,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":10196,"num_read_kb":27182,"num_write":429,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.3","version":"154'570","reported_seq":7311,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.889915+0000","last_change":"2024-09-17T23:31:55.866386+0000","last_active":"2024-09-17T23:31:57.889915+0000","last_peered":"2024-09-17T23:31:57.889915+0000","last_clean":"2024-09-17T23:31:57.889915+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:31:57.889915+0000","last_undegraded":"2024-09-17T23:31:57.889915+0000","last_fullsized":"2024-09-17T23:31:57.889915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:19:40.742854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018497380000000001,"stat_sum":{"num_bytes":17055232,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":24,"num_read":7271,"num_read_kb":29731,"num_write":591,"num_write_kb":9153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.2","version":"18'2","reported_seq":346,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867985+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:31:57.867985+0000","last_peered":"2024-09-17T23:31:57.867985+0000","last_clean":"2024-09-17T23:31:57.867985+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:31:57.867985+0000","last_undegraded":"2024-09-17T23:31:57.867985+0000","last_fullsized":"2024-09-17T23:31:57.867985+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"151'646","reported_seq":12000,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863325+0000","last_change":"2024-09-17T23:31:56.867801+0000","last_active":"2024-09-17T23:31:57.863325+0000","last_peered":"2024-09-17T23:31:57.863325+0000","last_clean":"2024-09-17T23:31:57.863325+0000","last_became_active":"2024-09-17T23:31:56.865272+0000","last_became_peered":"2024-09-17T23:31:56.865272+0000","last_unstale":"2024-09-17T23:31:57.863325+0000","last_undegraded":"2024-09-17T23:31:57.863325+0000","last_fullsized":"2024-09-17T23:31:57.863325+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:06:34.785652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000816283,"stat_sum":{"num_bytes":10143843,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":12767,"num_read_kb":56381,"num_write":711,"num_write_kb":10223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":297,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865836+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:31:57.865836+0000","last_peered":"2024-09-17T23:31:57.865836+0000","last_clean":"2024-09-17T23:31:57.865836+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:31:57.865836+0000","last_undegraded":"2024-09-17T23:31:57.865836+0000","last_fullsized":"2024-09-17T23:31:57.865836+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"151'857","reported_seq":8998,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863387+0000","last_change":"2024-09-17T23:31:56.868388+0000","last_active":"2024-09-17T23:31:57.863387+0000","last_peered":"2024-09-17T23:31:57.863387+0000","last_clean":"2024-09-17T23:31:57.863387+0000","last_became_active":"2024-09-17T23:31:56.865940+0000","last_became_peered":"2024-09-17T23:31:56.865940+0000","last_unstale":"2024-09-17T23:31:57.863387+0000","last_undegraded":"2024-09-17T23:31:57.863387+0000","last_fullsized":"2024-09-17T23:31:57.863387+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:25:08.005325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062989100000000004,"stat_sum":{"num_bytes":26297531,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11450,"num_read_kb":16034,"num_write":683,"num_write_kb":4230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863340+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:31:57.863340+0000","last_peered":"2024-09-17T23:31:57.863340+0000","last_clean":"2024-09-17T23:31:57.863340+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:31:57.863340+0000","last_undegraded":"2024-09-17T23:31:57.863340+0000","last_fullsized":"2024-09-17T23:31:57.863340+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"147'670","reported_seq":11205,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865337+0000","last_change":"2024-09-17T23:31:55.871630+0000","last_active":"2024-09-17T23:31:57.865337+0000","last_peered":"2024-09-17T23:31:57.865337+0000","last_clean":"2024-09-17T23:31:57.865337+0000","last_became_active":"2024-09-17T23:31:55.869290+0000","last_became_peered":"2024-09-17T23:31:55.869290+0000","last_unstale":"2024-09-17T23:31:57.865337+0000","last_undegraded":"2024-09-17T23:31:57.865337+0000","last_fullsized":"2024-09-17T23:31:57.865337+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:56:17.415710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086085100000000004,"stat_sum":{"num_bytes":23587937,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":11181,"num_read_kb":79280,"num_write":755,"num_write_kb":59941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"2.3","version":"16'1","reported_seq":215,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863712+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:31:57.863712+0000","last_peered":"2024-09-17T23:31:57.863712+0000","last_clean":"2024-09-17T23:31:57.863712+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:31:57.863712+0000","last_undegraded":"2024-09-17T23:31:57.863712+0000","last_fullsized":"2024-09-17T23:31:57.863712+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":381,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863392+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:31:57.863392+0000","last_peered":"2024-09-17T23:31:57.863392+0000","last_clean":"2024-09-17T23:31:57.863392+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:31:57.863392+0000","last_undegraded":"2024-09-17T23:31:57.863392+0000","last_fullsized":"2024-09-17T23:31:57.863392+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"150'432","reported_seq":14666,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.863475+0000","last_change":"2024-09-17T23:31:55.862141+0000","last_active":"2024-09-17T23:31:57.863475+0000","last_peered":"2024-09-17T23:31:57.863475+0000","last_clean":"2024-09-17T23:31:57.863475+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:31:57.863475+0000","last_undegraded":"2024-09-17T23:31:57.863475+0000","last_fullsized":"2024-09-17T23:31:57.863475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:03:52.265763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044238099999999999,"stat_sum":{"num_bytes":22973693,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":16,"num_read":15144,"num_read_kb":73922,"num_write":383,"num_write_kb":7864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.9","version":"153'697","reported_seq":11456,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867824+0000","last_change":"2024-09-17T23:31:55.869675+0000","last_active":"2024-09-17T23:31:57.867824+0000","last_peered":"2024-09-17T23:31:57.867824+0000","last_clean":"2024-09-17T23:31:57.867824+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:31:57.867824+0000","last_undegraded":"2024-09-17T23:31:57.867824+0000","last_fullsized":"2024-09-17T23:31:57.867824+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:31.783060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087999400000000005,"stat_sum":{"num_bytes":12578853,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":11139,"num_read_kb":39046,"num_write":813,"num_write_kb":25901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.a","version":"154'596","reported_seq":9750,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.882025+0000","last_change":"2024-09-17T23:31:55.866693+0000","last_active":"2024-09-17T23:31:57.882025+0000","last_peered":"2024-09-17T23:31:57.882025+0000","last_clean":"2024-09-17T23:31:57.882025+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:31:57.882025+0000","last_undegraded":"2024-09-17T23:31:57.882025+0000","last_fullsized":"2024-09-17T23:31:57.882025+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:48.541901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015858700000000001,"stat_sum":{"num_bytes":12512784,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":10955,"num_read_kb":53295,"num_write":595,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.b","version":"154'914","reported_seq":11472,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.872547+0000","last_change":"2024-09-17T23:31:57.872211+0000","last_active":"2024-09-17T23:31:57.872547+0000","last_peered":"2024-09-17T23:31:57.872547+0000","last_clean":"2024-09-17T23:31:57.872547+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:31:57.872547+0000","last_undegraded":"2024-09-17T23:31:57.872547+0000","last_fullsized":"2024-09-17T23:31:57.872547+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:11.321721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080631599999999996,"stat_sum":{"num_bytes":22895136,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":15013,"num_read_kb":61473,"num_write":890,"num_write_kb":11836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.c","version":"154'458","reported_seq":5329,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.918450+0000","last_change":"2024-09-17T23:31:55.865938+0000","last_active":"2024-09-17T23:31:57.918450+0000","last_peered":"2024-09-17T23:31:57.918450+0000","last_clean":"2024-09-17T23:31:57.918450+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:31:57.918450+0000","last_undegraded":"2024-09-17T23:31:57.918450+0000","last_fullsized":"2024-09-17T23:31:57.918450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:11.131387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019546559999999999,"stat_sum":{"num_bytes":9347621,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":14,"num_read":5960,"num_read_kb":36791,"num_write":451,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.d","version":"153'408","reported_seq":9657,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867994+0000","last_change":"2024-09-17T23:31:55.864311+0000","last_active":"2024-09-17T23:31:57.867994+0000","last_peered":"2024-09-17T23:31:57.867994+0000","last_clean":"2024-09-17T23:31:57.867994+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:31:57.867994+0000","last_undegraded":"2024-09-17T23:31:57.867994+0000","last_fullsized":"2024-09-17T23:31:57.867994+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:05:43.654480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001028153,"stat_sum":{"num_bytes":26217540,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":9068,"num_read_kb":42813,"num_write":455,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.e","version":"154'710","reported_seq":10052,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.885815+0000","last_change":"2024-09-17T23:31:56.867944+0000","last_active":"2024-09-17T23:31:57.885815+0000","last_peered":"2024-09-17T23:31:57.885815+0000","last_clean":"2024-09-17T23:31:57.885815+0000","last_became_active":"2024-09-17T23:31:56.865052+0000","last_became_peered":"2024-09-17T23:31:56.865052+0000","last_unstale":"2024-09-17T23:31:57.885815+0000","last_undegraded":"2024-09-17T23:31:57.885815+0000","last_fullsized":"2024-09-17T23:31:57.885815+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:10.451905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066576600000000004,"stat_sum":{"num_bytes":13903445,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":12,"num_read":10354,"num_read_kb":26378,"num_write":771,"num_write_kb":7195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.f","version":"150'358","reported_seq":4749,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868100+0000","last_change":"2024-09-17T23:31:55.871364+0000","last_active":"2024-09-17T23:31:57.868100+0000","last_peered":"2024-09-17T23:31:57.868100+0000","last_clean":"2024-09-17T23:31:57.868100+0000","last_became_active":"2024-09-17T23:31:55.869346+0000","last_became_peered":"2024-09-17T23:31:55.869346+0000","last_unstale":"2024-09-17T23:31:57.868100+0000","last_undegraded":"2024-09-17T23:31:57.868100+0000","last_fullsized":"2024-09-17T23:31:57.868100+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:05:21.800926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088462000000000002,"stat_sum":{"num_bytes":6846004,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":4817,"num_read_kb":27404,"num_write":373,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.10","version":"153'888","reported_seq":7793,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867815+0000","last_change":"2024-09-17T23:31:56.868103+0000","last_active":"2024-09-17T23:31:57.867815+0000","last_peered":"2024-09-17T23:31:57.867815+0000","last_clean":"2024-09-17T23:31:57.867815+0000","last_became_active":"2024-09-17T23:31:56.865636+0000","last_became_peered":"2024-09-17T23:31:56.865636+0000","last_unstale":"2024-09-17T23:31:57.867815+0000","last_undegraded":"2024-09-17T23:31:57.867815+0000","last_fullsized":"2024-09-17T23:31:57.867815+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:27.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060585100000000002,"stat_sum":{"num_bytes":21892689,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":9609,"num_read_kb":34061,"num_write":1199,"num_write_kb":24063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.11","version":"153'351","reported_seq":8487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.865638+0000","last_change":"2024-09-17T23:31:55.861212+0000","last_active":"2024-09-17T23:31:57.865638+0000","last_peered":"2024-09-17T23:31:57.865638+0000","last_clean":"2024-09-17T23:31:57.865638+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:31:57.865638+0000","last_undegraded":"2024-09-17T23:31:57.865638+0000","last_fullsized":"2024-09-17T23:31:57.865638+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:18.805689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002261079,"stat_sum":{"num_bytes":50728448,"num_objects":32,"num_object_clones":11,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":14,"num_read":8068,"num_read_kb":46907,"num_write":329,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.12","version":"153'427","reported_seq":5741,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.867745+0000","last_change":"2024-09-17T23:31:55.866468+0000","last_active":"2024-09-17T23:31:57.867745+0000","last_peered":"2024-09-17T23:31:57.867745+0000","last_clean":"2024-09-17T23:31:57.867745+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:31:57.867745+0000","last_undegraded":"2024-09-17T23:31:57.867745+0000","last_fullsized":"2024-09-17T23:31:57.867745+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:30:58.025203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001367851,"stat_sum":{"num_bytes":61967360,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":5018,"num_read_kb":38008,"num_write":480,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]},{"pgid":"3.13","version":"153'549","reported_seq":15105,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T23:31:57.868645+0000","last_change":"2024-09-17T23:31:55.866024+0000","last_active":"2024-09-17T23:31:57.868645+0000","last_peered":"2024-09-17T23:31:57.868645+0000","last_clean":"2024-09-17T23:31:57.868645+0000","last_became_active":"2024-09-17T23:31:43.947677+0000","last_became_peered":"2024-09-17T23:31:43.947677+0000","last_unstale":"2024-09-17T23:31:57.868645+0000","last_undegraded":"2024-09-17T23:31:57.868645+0000","last_fullsized":"2024-09-17T23:31:57.868645+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:38:06.492403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017458930000000001,"stat_sum":{"num_bytes":46105705,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":15068,"num_read_kb":74824,"num_write":704,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"17","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":773529987,"num_objects":944,"num_object_clones":217,"num_object_copies":1888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":944,"num_whiteouts":443,"num_read":309310,"num_read_kb":1323545,"num_write":19144,"num_write_kb":480717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7354,"num_bytes_recovered":259965128,"num_keys_recovered":1266,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23044096,"data_stored":171865640,"data_compressed":8295471,"data_compressed_allocated":17145856,"data_compressed_original":168584192,"omap_allocated":7598,"internal_metadata":0},"log_size":17650,"ondisk_log_size":17650,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500683,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46124,"kb_used_data":5236,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94325716,"statfs":{"total":96636764160,"available":96589533184,"internally_reserved":0,"allocated":5361664,"data_stored":30741505,"data_compressed":1207073,"data_compressed_allocated":2707456,"data_compressed_original":29506560,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640295,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50548,"kb_used_data":5464,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321292,"statfs":{"total":96636764160,"available":96585003008,"internally_reserved":0,"allocated":5595136,"data_stored":27751710,"data_compressed":1305723,"data_compressed_allocated":2613248,"data_compressed_original":26359296,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640295,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78364,"kb_used_data":6644,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94293476,"statfs":{"total":96636764160,"available":96556519424,"internally_reserved":0,"allocated":6803456,"data_stored":41139664,"data_compressed":1709488,"data_compressed_allocated":3661824,"data_compressed_original":39647232,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640295,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640296,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86788,"kb_used_data":6996,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94285052,"statfs":{"total":96636764160,"available":96547893248,"internally_reserved":0,"allocated":7163904,"data_stored":40189497,"data_compressed":1968342,"data_compressed_allocated":4087808,"data_compressed_original":38713344,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5308416,"data_stored":39361289,"data_compressed":1968306,"data_compressed_allocated":4083712,"data_compressed_original":38660096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":2250800,"data_compressed":129518,"data_compressed_allocated":262144,"data_compressed_original":2220032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4956160,"data_stored":40311594,"data_compressed":1709452,"data_compressed_allocated":3657728,"data_compressed_original":39593984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3706880,"data_stored":26386228,"data_compressed":1299945,"data_compressed_allocated":2576384,"data_compressed_original":25769472,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3448832,"data_stored":29370629,"data_compressed":1202131,"data_compressed_allocated":2670592,"data_compressed_original":28933120,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:00.628 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [4, 3, 5, 0] 2024-09-17T23:32:00.628 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.12', '4', '3', '5', '0'] 2024-09-17T23:32:00.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.12 4 3 5 0 2024-09-17T23:32:00.632 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:00.672 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:00.669+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:00.742 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:00.740+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:00.780 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.780 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:00.843 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:00.840+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:00.897 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:00.897 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:00.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:00.947 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:00.944+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:01.042 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:32:01.042 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.12 pg_upmap_items mapping to [4->3,5->0] 2024-09-17T23:32:01.045 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:01.042+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:01.061 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:01.061 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:01.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:01.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:32:01.147 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:01.144+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:01.225 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:01.225 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:01.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:01.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:01.239 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:01.247 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:01.244+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:01.256 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:01.274 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:01.291 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:01.291 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:01.308 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:01.309 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:01.342 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:01.342 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:01.348 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:01.345+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:01.362 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:01.362 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:01.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:01.456 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:01.453+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:01.519 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:01.520 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:01.520 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:01.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:01.570 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:01.567+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:01.651 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:01.649+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 182.20369840800001, 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:32:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:32:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:32:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.345862595, 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:32:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.34517030200001, 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.34515872099999, 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.34463147400001, 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:32:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.344597491, 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:32:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:32:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.344286899, 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:32:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.344260765, 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:32:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.34376285799999, 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:32:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:32:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:32:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:32:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.34261313299999, 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.34240553500001, 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:32:01.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:32:01.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.33278963399999, 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:32:01.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.33277939999999, 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:32:01.739 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:32:01.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:32:01.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:32:01.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.332769533, 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:32:01.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.332759385, 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:32:01.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:01.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.33228951699999, 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:32:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.332267401, 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:32:01.760 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:32:01.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.14659353600001, 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:32:01.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:32:01.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:32:01.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 35.772996909, 2024-09-17T23:32:01.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:01.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:32:01.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:01.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e136)", 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.810409561, 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031736553, 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23425 2024-09-17T23:32:01.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:01.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864730+0000", 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.485e-06 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864735+0000", 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864747+0000", 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1834999999999999e-05 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864758+0000", 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0431000000000001e-05 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864829+0000", 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1526000000000006e-05 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864837+0000", 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2509999999999994e-06 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867486+0000", 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002648376 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867499+0000", 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3553999999999999e-05 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896137+0000", 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.028637427 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896249+0000", 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011187 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:01.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896465+0000", 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021632800000000001 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:01.787 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:01.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:01.790 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:01.786+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:01.889 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:01.887+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:01.897 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:01.897 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:01.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:01.990 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:01.988+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:02.013 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:02.013 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:02.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:02.091 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:02.088+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:02.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.09924842500001, 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:02.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:32:02.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.09907548499999, 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:32:02.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:32:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:02.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:32:02.150 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.09903561499999, 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:32:02.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:32:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:32:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:32:02.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:32:02.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:32:02.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:32:02.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.80453138799999, 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:02.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:02.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:32:02.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:02.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:32:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:32:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:32:02.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:32:02.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:32:02.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:32:02.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:32:02.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:32:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:32:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:32:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:32:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:32:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:32:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:32:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:32:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:32:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:32:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:32:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:32:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:32:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.55682983899999, 2024-09-17T23:32:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:32:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:32:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:32:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:32:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:32:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:32:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:32:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:32:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:32:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:32:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:32:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:32:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:32:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:32:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:32:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:32:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:32:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:32:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.546897573, 2024-09-17T23:32:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:32:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:32:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:32:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:32:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:32:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:32:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:32:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:32:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:32:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:32:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.533670524, 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:32:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.993141368, 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:32:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.993119978, 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:32:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:32:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.993102849, 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:32:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.99308556699999, 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:32:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:32:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:32:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.993068538, 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:32:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:32:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:32:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:32:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:32:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:32:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:32:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:32:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:32:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:32:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:32:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:32:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:32:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.993035664, 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:32:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:32:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 121.992447958, 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 114.97065581699999, 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:32:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 114.970118629, 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:32:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 114.96963251299999, 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:32:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.89748991099999, 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:32:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:32:02.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:32:02.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.8974681, 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.88802523, 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:32:02.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:02.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:02.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:02.270 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:02.272 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:02.272 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:02.269+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:02.373 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:02.371+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:02.381 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:02.381 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:02.381 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:02.381 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:02.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:02.473 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:02.471+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:02.496 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:02.497 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:02.497 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:02.497 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:02.497 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:02.505 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:02.575 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:02.573+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:02.655 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:02.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.77733455200001, 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.656 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:32:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:32:02.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.662 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.761709422, 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:32:02.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:32:02.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.669 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.76167176600001, 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.670 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:32:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.761654335, 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:32:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:32:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:32:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.76161940399999, 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:32:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.76158798399999, 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:32:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:32:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.761571358, 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:32:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:32:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:32:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.761549664, 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:32:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 155.761537946, 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:32:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:32:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:32:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.802399427, 2024-09-17T23:32:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:32:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:32:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:32:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:32:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.79844426299999, 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:32:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.798403605, 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:32:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:32:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.798107958, 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:32:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:32:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.79807566300001, 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:32:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:32:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 101.797920852, 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:32:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.797634532000004, 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:32:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:32:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:32:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.797565871000003, 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:32:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:32:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 96.797523897000005, 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:32:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 41.797739874000001, 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:32:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:32:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:24856 3.b 3:d3034e8c:::rbd_header.1152ea45e50a:head [watch reconnect cookie 140329332260848 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.6464205379999992, 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0448234409999999, 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 24856 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005662+0000", 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9350000000000001e-06 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005669+0000", 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3470000000000004e-06 2024-09-17T23:32:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005683+0000", 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4419999999999999e-05 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005694+0000", 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0917999999999999e-05 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006157+0000", 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00046328800000000001 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006166+0000", 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041545+0000", 2024-09-17T23:32:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.035378916 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041630+0000", 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5628000000000001e-05 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045500+0000", 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038699910000000001 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045553+0000", 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2516000000000003e-05 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.050484+0000", 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049310329999999996 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:02.787 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:02.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:02.789 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:02.786+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:02.889 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:02.887+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:02.936 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:02.936 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:02.944 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:02.990 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:02.987+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:03.057 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:03.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:32:03.066 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:03.107 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:03.104+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:03.193 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:03.190+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:03.222 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.725866948, 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:32:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72584706800001, 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:32:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:32:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72582553000001, 2024-09-17T23:32:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:32:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:32:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.723932816, 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:32:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72392058599999, 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:32:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:32:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72391001700001, 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:32:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72390012899999, 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:32:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72388984700001, 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 145.72387395999999, 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:32:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:32:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 125.184948643, 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:32:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 125.182879257, 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:32:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 123.85101188900001, 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:32:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.36604721099999, 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:32:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.36601484099999, 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:32:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:32:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.365722464, 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:32:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:32:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.35989832600001, 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:32:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.359856823, 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:32:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.317118556, 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:32:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:32:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:32:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 102.317016215, 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:32:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:32:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 24.336702313, 2024-09-17T23:32:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:32:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:03.325 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:03.327 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:03.327 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:03.324+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:03.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:32:03.427 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:03.425+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:03.435 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:03.436 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:03.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:03.485 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:03.485 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:03.501 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:03.501 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:03.517 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:03.517 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:03.529 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:03.526+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:03.534 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:03.554 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:03.555 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:03.568 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:03.568 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:03.601 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:03.643 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:03.640+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:03.731 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:03.728+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:03.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:03.832 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:03.830+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:03.866 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:03.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:03.933 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:03.931+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:03.982 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:03.982 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:03.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:04.034 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:04.032+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:04.049 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:32:04.103 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.104 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:04.135 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:04.132+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:04.225 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.225 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:04.235 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:04.232+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:04.336 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:04.334+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:04.340 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.340 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:04.437 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:04.434+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:04.456 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.456 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:04.537 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:04.535+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:04.573 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.573 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:04.666 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:04.663+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:04.728 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.729 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:04.738 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:04.736+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:04.840 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:04.837+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:04.885 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:04.885 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:04.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:04.941 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:04.939+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:05.012 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.012 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.022 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:05.065 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:05.062+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:05.142 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.142 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.142 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:05.139+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:05.152 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:05.243 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:05.241+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:05.299 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.299 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:05.344 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:05.342+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:05.415 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.415 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:05.468 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:05.465+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:05.531 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.531 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:05.546 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:05.544+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:05.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:32:05.646 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:05.644+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:05.688 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.688 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.696 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:05.704 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:05.704 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:05.748 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:05.746+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:05.804 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.804 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.813 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:05.857 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:05.854+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:05.926 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:05.926 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:05.927 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:05.928 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:05.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:05.946 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:05.946 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:05.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:05.979 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:05.976+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:05.981 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:05.982 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:05.999 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:06.050 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:06.047+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:06.063 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:32:06.063 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:32:06.063 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T23:32:06.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T23:32:06.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:32:06.089 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:06.089 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.706194732999997, 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:32:06.090 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.092 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.093 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.094 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.103943680999997, 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.096 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.097 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.098 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:32:06.099 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:32:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.100 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:32:06.101 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:32:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:32:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:32:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 48.003311044, 2024-09-17T23:32:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:32:06.104 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.106 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.107 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:32:06.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.109 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.110 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.850783397999997, 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.113 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.114 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:32:06.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.116 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:32:06.117 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.850646159, 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.118 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:32:06.119 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:32:06.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:32:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:32:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.122 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.123 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.124 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 43.845488998999997, 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:32:06.127 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.128 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.130 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:32:06.131 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22907 3.0 3:07f1f9a0:::rbd_data.11524e94aa61.0000000000000007:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 35.110435013999997, 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053158520000000003, 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22907 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:32:06.133 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975829+0000", 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:32:06.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975838+0000", 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3260000000000002e-06 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975859+0000", 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0378999999999999e-05 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.135 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975870+0000", 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1488000000000001e-05 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975948+0000", 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199000000000001e-05 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976110+0000", 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000161977 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.136 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976234+0000", 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012316599999999999 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.977277+0000", 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010435679999999999 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981028+0000", 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037512130000000002 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981092+0000", 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3468999999999998e-05 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981143+0000", 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1017999999999998e-05 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22969 3.17 3:e879c454:::rbd_header.1152324fa224:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:32:06.138 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 34.197265573000003, 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058282270000000001, 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22969 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.139 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888998+0000", 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.140 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889005+0000", 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000004e-06 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889017+0000", 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2478e-05 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.141 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889029+0000", 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.168e-05 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889311+0000", 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028192400000000002 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889435+0000", 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000124046 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.142 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889886+0000", 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00045141000000000002 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.892147+0000", 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002260813 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894710+0000", 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025627710000000001 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.143 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894768+0000", 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8059000000000001e-05 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894825+0000", 2024-09-17T23:32:06.144 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6986e-05 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23135 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'391, mlcod=0'0)", 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 30.983053707, 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053043279999999996, 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.146 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103209+0000", 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103215+0000", 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999998e-06 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103222+0000", 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4919999999999996e-06 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.147 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103231+0000", 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6049999999999998e-06 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103277+0000", 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6641999999999998e-05 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103302+0000", 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4601e-05 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.148 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108442+0000", 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051404129999999999 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108513+0000", 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0192999999999997e-05 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24442 3.11 e145/138 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:head v 145'346, mlcod=140'344)", 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.010609434999999, 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054889680000000003, 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:06.150 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7499999999999999e-07 2024-09-17T23:32:06.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075660+0000", 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3430000000000001e-06 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075673+0000", 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2758000000000001e-05 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075680+0000", 2024-09-17T23:32:06.152 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0190000000000001e-06 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075731+0000", 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1872000000000001e-05 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075755+0000", 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3668000000000001e-05 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.153 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081093+0000", 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053383019999999996 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.154 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081142+0000", 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8231e-05 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24961 3.1a e150/139 3:5d9a7141:::rbd_data.1152ea45e50a.0000000000000015:head v 150'858, mlcod=146'854)", 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 11.900877216, 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055120910000000002, 2024-09-17T23:32:06.155 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4499999999999997e-07 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185391+0000", 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3820000000000003e-06 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185401+0000", 2024-09-17T23:32:06.157 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0271999999999999e-05 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185408+0000", 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8859999999999998e-06 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185462+0000", 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4234000000000003e-05 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185483+0000", 2024-09-17T23:32:06.158 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0747e-05 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.190854+0000", 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053711899999999996 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.190897+0000", 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:32:06.159 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25112 3.17 3:e827691a:::rbd_data.1152ce7c866.0000000000000001:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:32:06.160 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1350333549999991, 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069415060000000001, 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25112 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.162 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951232+0000", 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1700000000000001e-06 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951250+0000", 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7629999999999999e-05 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.163 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951257+0000", 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.401e-06 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952436+0000", 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011780849999999999 2024-09-17T23:32:06.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952728+0000", 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000292379 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952856+0000", 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012781000000000001 2024-09-17T23:32:06.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.954326+0000", 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014700410000000001 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958040+0000", 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037147080000000002 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958100+0000", 2024-09-17T23:32:06.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9796000000000001e-05 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958170+0000", 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9950000000000006e-05 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25113 3.17 3:ea59ff92:::rbd_data.1152ce7c866.0000000000000002:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1349270449999995, 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069436059999999997, 2024-09-17T23:32:06.167 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25113 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951336+0000", 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2099999999999996e-07 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951339+0000", 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3560000000000001e-06 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951351+0000", 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2422999999999999e-05 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951359+0000", 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.131e-06 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953211+0000", 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018522359999999999 2024-09-17T23:32:06.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953338+0000", 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012692899999999999 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953425+0000", 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7439000000000001e-05 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955541+0000", 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021159479999999999 2024-09-17T23:32:06.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958181+0000", 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026395770000000002 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958233+0000", 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2537999999999997e-05 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958279+0000", 2024-09-17T23:32:06.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5408000000000003e-05 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25115 3.1e e153/138 3:7ebe4efd:::rbd_data.1152ce7c866.0000000000000004:head v 153'526, mlcod=146'524)", 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1340953739999993, 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0059104259999999999, 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.173 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6100000000000002e-07 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952171+0000", 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8770000000000003e-06 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952178+0000", 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6579999999999996e-06 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952183+0000", 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5180000000000004e-06 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952263+0000", 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0242999999999997e-05 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952283+0000", 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9678999999999999e-05 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958014+0000", 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057307460000000001 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958077+0000", 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3343999999999995e-05 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25119 3.11 e153/138 3:8916a254:::rbd_data.1152ce7c866.0000000000000032:head v 153'351, mlcod=146'348)", 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1340660669999991, 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058297339999999996, 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:06.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1199999999999999e-07 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952204+0000", 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0290000000000005e-06 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952211+0000", 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7599999999999997e-06 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952216+0000", 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0069999999999997e-06 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952606+0000", 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038973799999999999 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952625+0000", 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9636999999999999e-05 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.957982+0000", 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053570220000000003 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958026+0000", 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3229000000000002e-05 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.182 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25124 3.e e153/152 3:71849f62:::rbd_data.1152ce7c866.0000000000000076:head v 153'706, mlcod=153'704)", 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1304112719999999, 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051704760000000002, 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955851+0000", 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9999999999999998e-07 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955861+0000", 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4690000000000003e-06 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955870+0000", 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8419999999999994e-06 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955874+0000", 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5410000000000002e-06 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955923+0000", 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9128000000000001e-05 2024-09-17T23:32:06.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955946+0000", 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3039000000000001e-05 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.960963+0000", 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0050171520000000004 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961021+0000", 2024-09-17T23:32:06.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7805e-05 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25125 3.10 e153/152 3:0a1561c8:::rbd_data.1152ce7c866.000000000000007c:head v 153'887, mlcod=0'0)", 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1302903010000005, 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.005055319, 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955972+0000", 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4299999999999998e-07 2024-09-17T23:32:06.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955976+0000", 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7450000000000001e-06 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955982+0000", 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.694e-06 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955992+0000", 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.078e-05 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.956037+0000", 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4280000000000003e-05 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.956055+0000", 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8357999999999999e-05 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.960970+0000", 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049147610000000001 2024-09-17T23:32:06.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961027+0000", 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7258e-05 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25134 3.17 3:e95af721:::rbd_data.1152ce7c866.000000000000012d:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.961311+0000", 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.1249510829999991, 2024-09-17T23:32:06.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049165739999999999, 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25134 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.193 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961311+0000", 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961311+0000", 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961312+0000", 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1900000000000001e-07 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961315+0000", 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.523e-06 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961326+0000", 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0507e-05 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961331+0000", 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1780000000000002e-06 2024-09-17T23:32:06.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961381+0000", 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9954999999999999e-05 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961567+0000", 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000186026 2024-09-17T23:32:06.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.961681+0000", 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000113621 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.962459+0000", 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00077861 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.197 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.966171+0000", 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003712042 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.966203+0000", 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1285999999999998e-05 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.966228+0000", 2024-09-17T23:32:06.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4907000000000001e-05 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25192 3.1a e155/139 3:5d1d36b7:::rbd_object_map.11526e6e10a0:head v 155'861, mlcod=154'859)", 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:58.873578+0000", 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 7.2126836909999996, 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0073394310000000004, 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873578+0000", 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873578+0000", 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873579+0000", 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3999999999999998e-07 2024-09-17T23:32:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873584+0000", 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3299999999999998e-06 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873601+0000", 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6714999999999998e-05 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873610+0000", 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7680000000000003e-06 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873669+0000", 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8576999999999998e-05 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873686+0000", 2024-09-17T23:32:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7280000000000001e-05 2024-09-17T23:32:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.880870+0000", 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0071837300000000002 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.880918+0000", 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7790999999999997e-05 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25726 3.17 3:edcf7f41:::rbd_header.115262cdab6:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158)", 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:32:02.041894+0000", 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.044367748, 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054082499999999999, 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:06.205 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25726 2024-09-17T23:32:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:06.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041894+0000", 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041894+0000", 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:06.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041896+0000", 2024-09-17T23:32:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8899999999999999e-06 2024-09-17T23:32:06.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041905+0000", 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3080000000000008e-06 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041922+0000", 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7774e-05 2024-09-17T23:32:06.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041932+0000", 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7690000000000008e-06 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.042010+0000", 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7461999999999998e-05 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.042018+0000", 2024-09-17T23:32:06.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3080000000000008e-06 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.044428+0000", 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024098639999999998 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.044554+0000", 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000126155 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.211 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.045062+0000", 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00050860800000000002 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.046499+0000", 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001436547 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.047183+0000", 2024-09-17T23:32:06.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00068358600000000002 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.047249+0000", 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6548e-05 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.047303+0000", 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3430999999999997e-05 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:06.213 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:06.214 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:06.215 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:06.212+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:06.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:06.316 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:06.313+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:06.326 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:06.326 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:06.326 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:06.326 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:06.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:06.360 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:06.360 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 162, 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "b662004c-f841-43da-8884-84d1805b0cbb", 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-17T23:28:46.056204+0000", 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-17T23:32:05.855799+0000", 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-17T23:31:11.038388+0000", 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-17T23:31:38.858088+0000", 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T23:32:06.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-17T23:32:06.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 6, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:28:58.406361+0000", 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:32:06.363 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-17T23:32:06.364 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "14", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:32:06.365 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:32:06.366 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:32:06.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.368 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-17T23:32:06.369 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:03.876212+0000", 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:32:06.370 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:32:06.371 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "18", 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 18, 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:32:06.372 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:32:06.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:32:06.374 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.9900000095367432, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.9900000095367432, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:32:06.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-17T23:29:14.532518+0000", 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-17T23:32:06.376 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "is_stretch_pool": false, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-17T23:32:06.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "162", 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T23:32:06.378 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 45, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 162, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-17T23:32:06.379 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-17T23:32:06.381 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.6099998950958252, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.6099998950958252, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.59311932325363159, 2024-09-17T23:32:06.382 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.5899999737739563, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.5899999737739563 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2c921380-aff6-4e43-bc78-48a393ed3673", 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:06.383 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 152, 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-17T23:32:06.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2793055955 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2793055955", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2793055955", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2793055955", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2793055955", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:06.388 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c95ad2cb-7bfe-4d1a-8280-640a59b6e73e", 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.0249176025390625, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 138, 2024-09-17T23:32:06.389 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.392 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.393 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2784091455 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2784091455", 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2784091455", 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2784091455", 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2784091455", 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "c7d869ca-7f8b-46df-882e-57da88ef6eda", 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.349456787109375, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 40, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:06.395 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.398 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 464841260 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.399 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/464841260", 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/464841260", 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/464841260", 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/464841260", 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:32:06.400 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ef4b1c62-ae14-4e5a-9914-d674364f3051", 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 153, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6800", 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6801", 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:06.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6802", 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6803", 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6806", 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6807", 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6804", 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6805", 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1436186932 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6801/1436186932", 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6803/1436186932", 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6807/1436186932", 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6805/1436186932", 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:32:06.406 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "fcd61898-fc45-4d4a-83b2-4f1bf9720309", 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 11, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 153, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6816", 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6817", 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6818", 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6819", 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:06.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6822", 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6823", 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:06.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6820", 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6821", 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 154081689 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6817/154081689", 2024-09-17T23:32:06.411 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6819/154081689", 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6823/154081689", 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6821/154081689", 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "1846216e-c369-4918-88be-300c9ee5397f", 2024-09-17T23:32:06.412 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.1843414306640625, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 11, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 93, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 107, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 150, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 94, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-17T23:32:06.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6808", 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6809", 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-17T23:32:06.414 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6810", 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6811", 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-17T23:32:06.415 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6814", 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6815", 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6812", 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.37:6813", 2024-09-17T23:32:06.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1359318569 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.37:6809/1359318569", 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.37:6811/1359318569", 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.37:6815/1359318569", 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.37:6813/1359318569", 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-17T23:32:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:55.062494+0000", 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 55177, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.641591+0000", 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.788407+0000", 2024-09-17T23:32:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.360450+0000", 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:06.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.895512+0000", 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-17T23:32:06.422 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-17T23:32:06.423 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-09-17T23:32:06.423 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-17T23:32:06.423 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-17T23:28:54.797662+0000", 2024-09-17T23:32:06.423 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 94 2024-09-17T23:32:06.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.12", 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: 2, 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-17T23:32:06.424 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [ 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.12", 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.425 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 5, 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 0 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-17T23:32:06.426 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 15, 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 6 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 15, 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.428 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [ 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 21, 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 13 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-17T23:32:06.429 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-17T23:32:06.430 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-17T23:32:06.430 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-17T23:32:06.430 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-17T23:32:06.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.430 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:06.432 INFO:tasks.thrashosds.thrasher:j is {'epoch': 162, 'fsid': 'b662004c-f841-43da-8884-84d1805b0cbb', 'created': '2024-09-17T23:28:46.056204+0000', 'modified': '2024-09-17T23:32:05.855799+0000', 'last_up_change': '2024-09-17T23:31:11.038388+0000', 'last_in_change': '2024-09-17T23:31:38.858088+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T23:28:58.406361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T23:29:03.876212+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 18, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T23:29:14.532518+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 162, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.609999895095825, 'score_stable': 2.609999895095825, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.5931193232536316, 'average_primary_affinity': 0.5899999737739563, 'average_primary_affinity_weighted': 0.5899999737739563}}], 'osds': [{'osd': 0, 'uuid': '2c921380-aff6-4e43-bc78-48a393ed3673', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 152, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2793055955}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2793055955}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2793055955}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2793055955}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2793055955}]}, 'public_addr': '172.21.15.31:6817/2793055955', 'cluster_addr': '172.21.15.31:6819/2793055955', 'heartbeat_back_addr': '172.21.15.31:6823/2793055955', 'heartbeat_front_addr': '172.21.15.31:6821/2793055955', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c95ad2cb-7bfe-4d1a-8280-640a59b6e73e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.0249176025390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2784091455}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2784091455}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2784091455}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2784091455}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2784091455}]}, 'public_addr': '172.21.15.31:6809/2784091455', 'cluster_addr': '172.21.15.31:6811/2784091455', 'heartbeat_back_addr': '172.21.15.31:6815/2784091455', 'heartbeat_front_addr': '172.21.15.31:6813/2784091455', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c7d869ca-7f8b-46df-882e-57da88ef6eda', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0.349456787109375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 464841260}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 464841260}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 464841260}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 464841260}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 464841260}]}, 'public_addr': '172.21.15.31:6801/464841260', 'cluster_addr': '172.21.15.31:6803/464841260', 'heartbeat_back_addr': '172.21.15.31:6807/464841260', 'heartbeat_front_addr': '172.21.15.31:6805/464841260', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ef4b1c62-ae14-4e5a-9914-d674364f3051', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6800', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6801', 'nonce': 1436186932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6802', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6803', 'nonce': 1436186932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6806', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6807', 'nonce': 1436186932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6804', 'nonce': 1436186932}, {'type': 'v1', 'addr': '172.21.15.37:6805', 'nonce': 1436186932}]}, 'public_addr': '172.21.15.37:6801/1436186932', 'cluster_addr': '172.21.15.37:6803/1436186932', 'heartbeat_back_addr': '172.21.15.37:6807/1436186932', 'heartbeat_front_addr': '172.21.15.37:6805/1436186932', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'fcd61898-fc45-4d4a-83b2-4f1bf9720309', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 153, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6816', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6817', 'nonce': 154081689}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6818', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6819', 'nonce': 154081689}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6822', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6823', 'nonce': 154081689}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6820', 'nonce': 154081689}, {'type': 'v1', 'addr': '172.21.15.37:6821', 'nonce': 154081689}]}, 'public_addr': '172.21.15.37:6817/154081689', 'cluster_addr': '172.21.15.37:6819/154081689', 'heartbeat_back_addr': '172.21.15.37:6823/154081689', 'heartbeat_front_addr': '172.21.15.37:6821/154081689', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1846216e-c369-4918-88be-300c9ee5397f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1843414306640625, 'last_clean_begin': 11, 'last_clean_end': 93, 'up_from': 107, 'up_thru': 150, 'down_at': 94, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6808', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6809', 'nonce': 1359318569}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6810', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6811', 'nonce': 1359318569}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6814', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6815', 'nonce': 1359318569}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6812', 'nonce': 1359318569}, {'type': 'v1', 'addr': '172.21.15.37:6813', 'nonce': 1359318569}]}, 'public_addr': '172.21.15.37:6809/1359318569', 'cluster_addr': '172.21.15.37:6811/1359318569', 'heartbeat_back_addr': '172.21.15.37:6815/1359318569', 'heartbeat_front_addr': '172.21.15.37:6813/1359318569', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:55.062494+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 55177, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.641591+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.788407+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.360450+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.895512+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T23:28:54.797662+0000', 'dead_epoch': 94}], 'pg_upmap': [{'pgid': '3.12', 'osds': [2, 1]}, {'pgid': '3.17', 'osds': [5, 4]}], 'pg_upmap_items': [{'pgid': '3.12', 'mappings': [{'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 15, 'length': 6}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 15, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 21, 'length': 13}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T23:32:06.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:06.433 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:06.429+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:06.532 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:06.530+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:06.537 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:06.537 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:06.538 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:06.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:06.634 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:06.631+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:06.658 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:06.658 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:06.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:06.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:06.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:32:06.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:32:06.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.81309405499999, 2024-09-17T23:32:06.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:32:06.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:32:06.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:32:06.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.803521956, 2024-09-17T23:32:06.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:32:06.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:32:06.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:32:06.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79999144999999, 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:32:06.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:32:06.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:32:06.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.798496744, 2024-09-17T23:32:06.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:32:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:32:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:32:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:32:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79831382499999, 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:32:06.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:32:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:32:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:32:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:32:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:32:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:32:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:32:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:32:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:32:06.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:32:06.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79820032400001, 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:32:06.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:32:06.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:32:06.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:32:06.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79817621700001, 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:32:06.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:32:06.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:32:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:32:06.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:32:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:32:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:32:06.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.798101886, 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:32:06.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79807833700001, 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:32:06.713 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:32:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:32:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79490713600001, 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:32:06.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.79049879999999, 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:32:06.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.76322669300001, 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:32:06.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:32:06.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:32:06.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.754044541, 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:32:06.735 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:32:06.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:32:06.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:06.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.754013048, 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:32:06.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:32:06.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.753987356, 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:32:06.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.753842804, 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:32:06.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:32:06.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.752274687, 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:32:06.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:32:06.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.75207734, 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:32:06.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:32:06.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:32:06.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.752051025, 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:32:06.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:32:06.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.752022682, 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:06.773 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:32:06.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:32:06.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:06.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:06.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:06.778 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:06.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:06.781 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:06.783 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-17T23:32:06.393418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567739974,"num_objects":777,"num_object_clones":189,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":428,"num_read":310783,"num_read_kb":1324804,"num_write":19608,"num_write_kb":483203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7367,"num_bytes_recovered":260555591,"num_keys_recovered":1266,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18147,"ondisk_log_size":18147,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":244736,"kb_used_data":19856,"kb_used_omap":24,"kb_used_meta":224551,"kb_avail":377242624,"statfs":{"total":386547056640,"available":386296446976,"internally_reserved":0,"allocated":20332544,"data_stored":109663442,"data_compressed":3710279,"data_compressed_allocated":9191424,"data_compressed_original":104544256,"omap_allocated":24578,"internal_metadata":229941246},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-206380400,"num_objects":-171,"num_object_clones":-28,"num_object_copies":-342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-171,"num_whiteouts":-15,"num_read":1357,"num_read_kb":1159,"num_write":303,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000893"},"pg_stats":[{"pgid":"3.1f","version":"157'420","reported_seq":5796,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.047343+0000","last_change":"2024-09-17T23:32:03.047685+0000","last_active":"2024-09-17T23:32:03.047343+0000","last_peered":"2024-09-17T23:32:03.047343+0000","last_clean":"2024-09-17T23:32:03.047343+0000","last_became_active":"2024-09-17T23:31:55.865083+0000","last_became_peered":"2024-09-17T23:31:55.865083+0000","last_unstale":"2024-09-17T23:32:03.047343+0000","last_undegraded":"2024-09-17T23:32:03.047343+0000","last_fullsized":"2024-09-17T23:32:03.047343+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:37:41.559947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001341809,"stat_sum":{"num_bytes":28607028,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":7434,"num_read_kb":66454,"num_write":456,"num_write_kb":20585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.1e","version":"159'532","reported_seq":8943,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.203504+0000","last_change":"2024-09-17T23:32:03.048092+0000","last_active":"2024-09-17T23:32:03.203504+0000","last_peered":"2024-09-17T23:32:03.203504+0000","last_clean":"2024-09-17T23:32:03.203504+0000","last_became_active":"2024-09-17T23:31:42.948126+0000","last_became_peered":"2024-09-17T23:31:42.948126+0000","last_unstale":"2024-09-17T23:32:03.203504+0000","last_undegraded":"2024-09-17T23:32:03.203504+0000","last_fullsized":"2024-09-17T23:32:03.203504+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:05.965814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001430506,"stat_sum":{"num_bytes":76732069,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":16,"num_read":9327,"num_read_kb":55087,"num_write":541,"num_write_kb":19041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.1d","version":"158'357","reported_seq":5842,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.050855+0000","last_change":"2024-09-17T23:32:03.051139+0000","last_active":"2024-09-17T23:32:03.050855+0000","last_peered":"2024-09-17T23:32:03.050855+0000","last_clean":"2024-09-17T23:32:03.050855+0000","last_became_active":"2024-09-17T23:31:42.947064+0000","last_became_peered":"2024-09-17T23:31:42.947064+0000","last_unstale":"2024-09-17T23:32:03.050855+0000","last_undegraded":"2024-09-17T23:32:03.050855+0000","last_fullsized":"2024-09-17T23:32:03.050855+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:28.650049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016983269999999999,"stat_sum":{"num_bytes":12722688,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":4805,"num_read_kb":41204,"num_write":328,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.1c","version":"159'441","reported_seq":9188,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.169762+0000","last_change":"2024-09-17T23:32:03.051600+0000","last_active":"2024-09-17T23:32:03.169762+0000","last_peered":"2024-09-17T23:32:03.169762+0000","last_clean":"2024-09-17T23:32:03.169762+0000","last_became_active":"2024-09-17T23:31:56.864933+0000","last_became_peered":"2024-09-17T23:31:56.864933+0000","last_unstale":"2024-09-17T23:32:03.169762+0000","last_undegraded":"2024-09-17T23:32:03.169762+0000","last_fullsized":"2024-09-17T23:32:03.169762+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:30.529648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021224120000000002,"stat_sum":{"num_bytes":2916352,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":8483,"num_read_kb":40229,"num_write":536,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.1b","version":"158'435","reported_seq":19588,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.046973+0000","last_change":"2024-09-17T23:32:03.047224+0000","last_active":"2024-09-17T23:32:03.046973+0000","last_peered":"2024-09-17T23:32:03.046973+0000","last_clean":"2024-09-17T23:32:03.046973+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:03.046973+0000","last_undegraded":"2024-09-17T23:32:03.046973+0000","last_fullsized":"2024-09-17T23:32:03.046973+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:12.848456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010259749999999999,"stat_sum":{"num_bytes":27992337,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":21,"num_read":18892,"num_read_kb":58432,"num_write":474,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.1a","version":"157'875","reported_seq":7521,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.185457+0000","last_change":"2024-09-17T23:32:03.052626+0000","last_active":"2024-09-17T23:32:03.185457+0000","last_peered":"2024-09-17T23:32:03.185457+0000","last_clean":"2024-09-17T23:32:03.185457+0000","last_became_active":"2024-09-17T23:31:43.945744+0000","last_became_peered":"2024-09-17T23:31:43.945744+0000","last_unstale":"2024-09-17T23:32:03.185457+0000","last_undegraded":"2024-09-17T23:32:03.185457+0000","last_fullsized":"2024-09-17T23:32:03.185457+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:28:02.638962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054411399999999995,"stat_sum":{"num_bytes":21420067,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":9,"num_read":12001,"num_read_kb":30900,"num_write":1208,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":7389463,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.19","version":"158'808","reported_seq":6566,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.053501+0000","last_change":"2024-09-17T23:32:03.053747+0000","last_active":"2024-09-17T23:32:03.053501+0000","last_peered":"2024-09-17T23:32:03.053501+0000","last_clean":"2024-09-17T23:32:03.053501+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:03.053501+0000","last_undegraded":"2024-09-17T23:32:03.053501+0000","last_fullsized":"2024-09-17T23:32:03.053501+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:25.240675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001166119,"stat_sum":{"num_bytes":12962816,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":8538,"num_read_kb":17541,"num_write":916,"num_write_kb":3496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.18","version":"158'440","reported_seq":4312,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.046928+0000","last_change":"2024-09-17T23:32:03.047260+0000","last_active":"2024-09-17T23:32:03.046928+0000","last_peered":"2024-09-17T23:32:03.046928+0000","last_clean":"2024-09-17T23:32:03.046928+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:03.046928+0000","last_undegraded":"2024-09-17T23:32:03.046928+0000","last_fullsized":"2024-09-17T23:32:03.046928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:35:29.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156511,"stat_sum":{"num_bytes":8561189,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":16,"num_read":4989,"num_read_kb":22192,"num_write":436,"num_write_kb":20028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.17","version":"160'655","reported_seq":8190,"reported_epoch":161,"state":"active+clean+snaptrim","last_fresh":"2024-09-17T23:32:05.065088+0000","last_change":"2024-09-17T23:32:05.065122+0000","last_active":"2024-09-17T23:32:05.065088+0000","last_peered":"2024-09-17T23:32:05.065088+0000","last_clean":"2024-09-17T23:32:05.065088+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:05.065088+0000","last_undegraded":"2024-09-17T23:32:05.065088+0000","last_fullsized":"2024-09-17T23:32:05.065088+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:31:45.770844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083827299999999995,"stat_sum":{"num_bytes":30252070,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":11076,"num_read_kb":29085,"num_write":556,"num_write_kb":13275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"11"}]},{"pgid":"3.16","version":"156'353","reported_seq":4329,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.047587+0000","last_change":"2024-09-17T23:32:03.047875+0000","last_active":"2024-09-17T23:32:03.047587+0000","last_peered":"2024-09-17T23:32:03.047587+0000","last_clean":"2024-09-17T23:32:03.047587+0000","last_became_active":"2024-09-17T23:31:55.866071+0000","last_became_peered":"2024-09-17T23:31:55.866071+0000","last_unstale":"2024-09-17T23:32:03.047587+0000","last_undegraded":"2024-09-17T23:32:03.047587+0000","last_fullsized":"2024-09-17T23:32:03.047587+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:04.841906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013710619999999999,"stat_sum":{"num_bytes":12459568,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":4431,"num_read_kb":31183,"num_write":360,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.15","version":"158'621","reported_seq":12915,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052490+0000","last_change":"2024-09-17T23:32:03.052763+0000","last_active":"2024-09-17T23:32:03.052490+0000","last_peered":"2024-09-17T23:32:03.052490+0000","last_clean":"2024-09-17T23:32:03.052490+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:03.052490+0000","last_undegraded":"2024-09-17T23:32:03.052490+0000","last_fullsized":"2024-09-17T23:32:03.052490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:25:42.628716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027834589999999998,"stat_sum":{"num_bytes":25166848,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":14723,"num_read_kb":31747,"num_write":651,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.14","version":"159'599","reported_seq":8109,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.206446+0000","last_change":"2024-09-17T23:32:03.052067+0000","last_active":"2024-09-17T23:32:03.206446+0000","last_peered":"2024-09-17T23:32:03.206446+0000","last_clean":"2024-09-17T23:32:03.206446+0000","last_became_active":"2024-09-17T23:30:28.866310+0000","last_became_peered":"2024-09-17T23:30:28.866310+0000","last_unstale":"2024-09-17T23:32:03.206446+0000","last_undegraded":"2024-09-17T23:32:03.206446+0000","last_fullsized":"2024-09-17T23:32:03.206446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:30:43.544500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023189399999999998,"stat_sum":{"num_bytes":1736720,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":8781,"num_read_kb":33881,"num_write":502,"num_write_kb":18378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.7","version":"0'0","reported_seq":253,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.049501+0000","last_change":"2024-09-17T23:30:27.861326+0000","last_active":"2024-09-17T23:32:03.049501+0000","last_peered":"2024-09-17T23:32:03.049501+0000","last_clean":"2024-09-17T23:32:03.049501+0000","last_became_active":"2024-09-17T23:30:27.860975+0000","last_became_peered":"2024-09-17T23:30:27.860975+0000","last_unstale":"2024-09-17T23:32:03.049501+0000","last_undegraded":"2024-09-17T23:32:03.049501+0000","last_fullsized":"2024-09-17T23:32:03.049501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:41:14.615862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.6","version":"159'617","reported_seq":5082,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.053218+0000","last_change":"2024-09-17T23:32:03.053218+0000","last_active":"2024-09-17T23:32:03.053218+0000","last_peered":"2024-09-17T23:32:03.053218+0000","last_clean":"2024-09-17T23:32:03.053218+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:03.053218+0000","last_undegraded":"2024-09-17T23:32:03.053218+0000","last_fullsized":"2024-09-17T23:32:03.053218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:56:51.345307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090891099999999999,"stat_sum":{"num_bytes":830551,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":5974,"num_read_kb":22618,"num_write":877,"num_write_kb":9102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.6","version":"0'0","reported_seq":302,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052160+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:03.052160+0000","last_peered":"2024-09-17T23:32:03.052160+0000","last_clean":"2024-09-17T23:32:03.052160+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:03.052160+0000","last_undegraded":"2024-09-17T23:32:03.052160+0000","last_fullsized":"2024-09-17T23:32:03.052160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'424","reported_seq":12540,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.205445+0000","last_change":"2024-09-17T23:32:03.053820+0000","last_active":"2024-09-17T23:32:03.205445+0000","last_peered":"2024-09-17T23:32:03.205445+0000","last_clean":"2024-09-17T23:32:03.205445+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:03.205445+0000","last_undegraded":"2024-09-17T23:32:03.205445+0000","last_fullsized":"2024-09-17T23:32:03.205445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:03:36.114763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000956652,"stat_sum":{"num_bytes":4456448,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":13128,"num_read_kb":60562,"num_write":428,"num_write_kb":30574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052445+0000","last_change":"2024-09-17T23:31:39.869108+0000","last_active":"2024-09-17T23:32:03.052445+0000","last_peered":"2024-09-17T23:32:03.052445+0000","last_clean":"2024-09-17T23:32:03.052445+0000","last_became_active":"2024-09-17T23:31:39.868893+0000","last_became_peered":"2024-09-17T23:31:39.868893+0000","last_unstale":"2024-09-17T23:32:03.052445+0000","last_undegraded":"2024-09-17T23:32:03.052445+0000","last_fullsized":"2024-09-17T23:32:03.052445+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:48:31.903251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"155'385","reported_seq":7260,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052709+0000","last_change":"2024-09-17T23:32:03.053087+0000","last_active":"2024-09-17T23:32:03.052709+0000","last_peered":"2024-09-17T23:32:03.052709+0000","last_clean":"2024-09-17T23:32:03.052709+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:03.052709+0000","last_undegraded":"2024-09-17T23:32:03.052709+0000","last_fullsized":"2024-09-17T23:32:03.052709+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:08:03.293053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090539400000000001,"stat_sum":{"num_bytes":9269278,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":22,"num_read":4593,"num_read_kb":19709,"num_write":436,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.046092+0000","last_change":"2024-09-17T23:31:54.009886+0000","last_active":"2024-09-17T23:32:03.046092+0000","last_peered":"2024-09-17T23:32:03.046092+0000","last_clean":"2024-09-17T23:32:03.046092+0000","last_became_active":"2024-09-17T23:31:54.009591+0000","last_became_peered":"2024-09-17T23:31:54.009591+0000","last_unstale":"2024-09-17T23:32:03.046092+0000","last_undegraded":"2024-09-17T23:32:03.046092+0000","last_fullsized":"2024-09-17T23:32:03.046092+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:40.324116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"155'373","reported_seq":10507,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.047928+0000","last_change":"2024-09-17T23:32:03.048277+0000","last_active":"2024-09-17T23:32:03.047928+0000","last_peered":"2024-09-17T23:32:03.047928+0000","last_clean":"2024-09-17T23:32:03.047928+0000","last_became_active":"2024-09-17T23:30:21.852159+0000","last_became_peered":"2024-09-17T23:30:21.852159+0000","last_unstale":"2024-09-17T23:32:03.047928+0000","last_undegraded":"2024-09-17T23:32:03.047928+0000","last_fullsized":"2024-09-17T23:32:03.047928+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:09.162174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015834829999999999,"stat_sum":{"num_bytes":17661472,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":10196,"num_read_kb":27182,"num_write":431,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.3","version":"159'594","reported_seq":7409,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.175736+0000","last_change":"2024-09-17T23:32:03.052010+0000","last_active":"2024-09-17T23:32:03.175736+0000","last_peered":"2024-09-17T23:32:03.175736+0000","last_clean":"2024-09-17T23:32:03.175736+0000","last_became_active":"2024-09-17T23:30:21.853875+0000","last_became_peered":"2024-09-17T23:30:21.853875+0000","last_unstale":"2024-09-17T23:32:03.175736+0000","last_undegraded":"2024-09-17T23:32:03.175736+0000","last_fullsized":"2024-09-17T23:32:03.175736+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:52.366239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002328545,"stat_sum":{"num_bytes":12533248,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":22,"num_read":7369,"num_read_kb":29819,"num_write":611,"num_write_kb":9168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.2","version":"18'2","reported_seq":356,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.049694+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:03.049694+0000","last_peered":"2024-09-17T23:32:03.049694+0000","last_clean":"2024-09-17T23:32:03.049694+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:03.049694+0000","last_undegraded":"2024-09-17T23:32:03.049694+0000","last_fullsized":"2024-09-17T23:32:03.049694+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"158'647","reported_seq":12038,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052619+0000","last_change":"2024-09-17T23:32:03.052619+0000","last_active":"2024-09-17T23:32:03.052619+0000","last_peered":"2024-09-17T23:32:03.052619+0000","last_clean":"2024-09-17T23:32:03.052619+0000","last_became_active":"2024-09-17T23:31:56.865272+0000","last_became_peered":"2024-09-17T23:31:56.865272+0000","last_unstale":"2024-09-17T23:32:03.052619+0000","last_undegraded":"2024-09-17T23:32:03.052619+0000","last_fullsized":"2024-09-17T23:32:03.052619+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:13:20.752972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054196400000000003,"stat_sum":{"num_bytes":3065955,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":12771,"num_read_kb":56383,"num_write":712,"num_write_kb":10223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.1","version":"0'0","reported_seq":307,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.046453+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:03.046453+0000","last_peered":"2024-09-17T23:32:03.046453+0000","last_clean":"2024-09-17T23:32:03.046453+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:03.046453+0000","last_undegraded":"2024-09-17T23:32:03.046453+0000","last_fullsized":"2024-09-17T23:32:03.046453+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"158'884","reported_seq":9132,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.053146+0000","last_change":"2024-09-17T23:32:03.053451+0000","last_active":"2024-09-17T23:32:03.053146+0000","last_peered":"2024-09-17T23:32:03.053146+0000","last_clean":"2024-09-17T23:32:03.053146+0000","last_became_active":"2024-09-17T23:31:56.865940+0000","last_became_peered":"2024-09-17T23:31:56.865940+0000","last_unstale":"2024-09-17T23:32:03.053146+0000","last_undegraded":"2024-09-17T23:32:03.053146+0000","last_fullsized":"2024-09-17T23:32:03.053146+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:27.242846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00109614,"stat_sum":{"num_bytes":25872422,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":11620,"num_read_kb":16181,"num_write":703,"num_write_kb":4236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.0","version":"0'0","reported_seq":171,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.051966+0000","last_change":"2024-09-17T23:31:26.860273+0000","last_active":"2024-09-17T23:32:03.051966+0000","last_peered":"2024-09-17T23:32:03.051966+0000","last_clean":"2024-09-17T23:32:03.051966+0000","last_became_active":"2024-09-17T23:31:26.859967+0000","last_became_peered":"2024-09-17T23:31:26.859967+0000","last_unstale":"2024-09-17T23:32:03.051966+0000","last_undegraded":"2024-09-17T23:32:03.051966+0000","last_fullsized":"2024-09-17T23:32:03.051966+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:29:27.039912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"158'680","reported_seq":11252,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.160313+0000","last_change":"2024-09-17T23:32:03.046896+0000","last_active":"2024-09-17T23:32:03.160313+0000","last_peered":"2024-09-17T23:32:03.160313+0000","last_clean":"2024-09-17T23:32:03.160313+0000","last_became_active":"2024-09-17T23:31:55.869290+0000","last_became_peered":"2024-09-17T23:31:55.869290+0000","last_unstale":"2024-09-17T23:32:03.160313+0000","last_undegraded":"2024-09-17T23:32:03.160313+0000","last_fullsized":"2024-09-17T23:32:03.160313+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:34:57.882068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077096399999999996,"stat_sum":{"num_bytes":11922529,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":11184,"num_read_kb":79281,"num_write":763,"num_write_kb":59941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"2.3","version":"16'1","reported_seq":225,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052155+0000","last_change":"2024-09-17T23:31:39.879014+0000","last_active":"2024-09-17T23:32:03.052155+0000","last_peered":"2024-09-17T23:32:03.052155+0000","last_clean":"2024-09-17T23:32:03.052155+0000","last_became_active":"2024-09-17T23:31:39.873587+0000","last_became_peered":"2024-09-17T23:31:39.873587+0000","last_unstale":"2024-09-17T23:32:03.052155+0000","last_undegraded":"2024-09-17T23:32:03.052155+0000","last_fullsized":"2024-09-17T23:32:03.052155+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:24:54.855470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":391,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052073+0000","last_change":"2024-09-17T23:31:42.946545+0000","last_active":"2024-09-17T23:32:03.052073+0000","last_peered":"2024-09-17T23:32:03.052073+0000","last_clean":"2024-09-17T23:32:03.052073+0000","last_became_active":"2024-09-17T23:31:42.946166+0000","last_became_peered":"2024-09-17T23:31:42.946166+0000","last_unstale":"2024-09-17T23:32:03.052073+0000","last_undegraded":"2024-09-17T23:32:03.052073+0000","last_fullsized":"2024-09-17T23:32:03.052073+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.145297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"159'452","reported_seq":14727,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.145220+0000","last_change":"2024-09-17T23:32:03.053579+0000","last_active":"2024-09-17T23:32:03.145220+0000","last_peered":"2024-09-17T23:32:03.145220+0000","last_clean":"2024-09-17T23:32:03.145220+0000","last_became_active":"2024-09-17T23:30:21.855636+0000","last_became_peered":"2024-09-17T23:30:21.855636+0000","last_unstale":"2024-09-17T23:32:03.145220+0000","last_undegraded":"2024-09-17T23:32:03.145220+0000","last_fullsized":"2024-09-17T23:32:03.145220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:45:52.876599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012325000000000001,"stat_sum":{"num_bytes":19113561,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":15178,"num_read_kb":73947,"num_write":403,"num_write_kb":7869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.9","version":"158'704","reported_seq":11498,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.049286+0000","last_change":"2024-09-17T23:32:03.049890+0000","last_active":"2024-09-17T23:32:03.049286+0000","last_peered":"2024-09-17T23:32:03.049286+0000","last_clean":"2024-09-17T23:32:03.049286+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:03.049286+0000","last_undegraded":"2024-09-17T23:32:03.049286+0000","last_fullsized":"2024-09-17T23:32:03.049286+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:38.990010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000862439,"stat_sum":{"num_bytes":1781797,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":13,"num_read":11139,"num_read_kb":39046,"num_write":818,"num_write_kb":25901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.a","version":"159'605","reported_seq":9794,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.139999+0000","last_change":"2024-09-17T23:32:03.051157+0000","last_active":"2024-09-17T23:32:03.139999+0000","last_peered":"2024-09-17T23:32:03.139999+0000","last_clean":"2024-09-17T23:32:03.139999+0000","last_became_active":"2024-09-17T23:31:12.059854+0000","last_became_peered":"2024-09-17T23:31:12.059854+0000","last_unstale":"2024-09-17T23:32:03.139999+0000","last_undegraded":"2024-09-17T23:32:03.139999+0000","last_fullsized":"2024-09-17T23:32:03.139999+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:34:27.248544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017034470000000001,"stat_sum":{"num_bytes":7397904,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":16,"num_read":10957,"num_read_kb":53297,"num_write":602,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.b","version":"157'921","reported_seq":11513,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.047283+0000","last_change":"2024-09-17T23:32:03.047533+0000","last_active":"2024-09-17T23:32:03.047283+0000","last_peered":"2024-09-17T23:32:03.047283+0000","last_clean":"2024-09-17T23:32:03.047283+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:03.047283+0000","last_undegraded":"2024-09-17T23:32:03.047283+0000","last_fullsized":"2024-09-17T23:32:03.047283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:43:19.393267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328203,"stat_sum":{"num_bytes":20077088,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":15013,"num_read_kb":61473,"num_write":893,"num_write_kb":11836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.c","version":"157'465","reported_seq":5370,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052148+0000","last_change":"2024-09-17T23:32:03.052410+0000","last_active":"2024-09-17T23:32:03.052148+0000","last_peered":"2024-09-17T23:32:03.052148+0000","last_clean":"2024-09-17T23:32:03.052148+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:03.052148+0000","last_undegraded":"2024-09-17T23:32:03.052148+0000","last_fullsized":"2024-09-17T23:32:03.052148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:29:37.607035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00257362,"stat_sum":{"num_bytes":3720229,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":5960,"num_read_kb":36791,"num_write":454,"num_write_kb":5975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.d","version":"157'417","reported_seq":9700,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.052079+0000","last_change":"2024-09-17T23:32:03.052414+0000","last_active":"2024-09-17T23:32:03.052079+0000","last_peered":"2024-09-17T23:32:03.052079+0000","last_clean":"2024-09-17T23:32:03.052079+0000","last_became_active":"2024-09-17T23:31:42.950616+0000","last_became_peered":"2024-09-17T23:31:42.950616+0000","last_unstale":"2024-09-17T23:32:03.052079+0000","last_undegraded":"2024-09-17T23:32:03.052079+0000","last_fullsized":"2024-09-17T23:32:03.052079+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:38:03.222755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026849159999999999,"stat_sum":{"num_bytes":23054916,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":9068,"num_read_kb":42813,"num_write":458,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.e","version":"156'712","reported_seq":10109,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.186066+0000","last_change":"2024-09-17T23:32:03.050765+0000","last_active":"2024-09-17T23:32:03.186066+0000","last_peered":"2024-09-17T23:32:03.186066+0000","last_clean":"2024-09-17T23:32:03.186066+0000","last_became_active":"2024-09-17T23:31:56.865052+0000","last_became_peered":"2024-09-17T23:31:56.865052+0000","last_unstale":"2024-09-17T23:32:03.186066+0000","last_undegraded":"2024-09-17T23:32:03.186066+0000","last_fullsized":"2024-09-17T23:32:03.186066+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:34:16.952532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014920059999999999,"stat_sum":{"num_bytes":9836117,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":10424,"num_read_kb":26438,"num_write":773,"num_write_kb":7195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.f","version":"155'360","reported_seq":4786,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.050481+0000","last_change":"2024-09-17T23:32:03.050773+0000","last_active":"2024-09-17T23:32:03.050481+0000","last_peered":"2024-09-17T23:32:03.050481+0000","last_clean":"2024-09-17T23:32:03.050481+0000","last_became_active":"2024-09-17T23:31:55.869346+0000","last_became_peered":"2024-09-17T23:31:55.869346+0000","last_unstale":"2024-09-17T23:32:03.050481+0000","last_undegraded":"2024-09-17T23:32:03.050481+0000","last_fullsized":"2024-09-17T23:32:03.050481+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:05:25.032267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001478732,"stat_sum":{"num_bytes":6452788,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":4817,"num_read_kb":27404,"num_write":375,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.10","version":"158'905","reported_seq":7849,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.049919+0000","last_change":"2024-09-17T23:32:03.050311+0000","last_active":"2024-09-17T23:32:03.049919+0000","last_peered":"2024-09-17T23:32:03.049919+0000","last_clean":"2024-09-17T23:32:03.049919+0000","last_became_active":"2024-09-17T23:31:56.865636+0000","last_became_peered":"2024-09-17T23:31:56.865636+0000","last_unstale":"2024-09-17T23:32:03.049919+0000","last_undegraded":"2024-09-17T23:32:03.049919+0000","last_fullsized":"2024-09-17T23:32:03.049919+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:02.878284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012504409999999999,"stat_sum":{"num_bytes":1718310,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":9640,"num_read_kb":34085,"num_write":1218,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":459,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.11","version":"158'360","reported_seq":8530,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.046401+0000","last_change":"2024-09-17T23:32:03.046870+0000","last_active":"2024-09-17T23:32:03.046401+0000","last_peered":"2024-09-17T23:32:03.046401+0000","last_clean":"2024-09-17T23:32:03.046401+0000","last_became_active":"2024-09-17T23:31:42.943531+0000","last_became_peered":"2024-09-17T23:31:42.943531+0000","last_unstale":"2024-09-17T23:32:03.046401+0000","last_undegraded":"2024-09-17T23:32:03.046401+0000","last_fullsized":"2024-09-17T23:32:03.046401+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:15.602283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077220000000000001,"stat_sum":{"num_bytes":38057472,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":8068,"num_read_kb":46907,"num_write":334,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":338,"num_bytes_recovered":18644584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.12","version":"153'427","reported_seq":5776,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.050008+0000","last_change":"2024-09-17T23:32:03.050421+0000","last_active":"2024-09-17T23:32:03.050008+0000","last_peered":"2024-09-17T23:32:03.050008+0000","last_clean":"2024-09-17T23:32:03.050008+0000","last_became_active":"2024-09-17T23:31:37.869733+0000","last_became_peered":"2024-09-17T23:31:37.869733+0000","last_unstale":"2024-09-17T23:32:03.050008+0000","last_undegraded":"2024-09-17T23:32:03.050008+0000","last_fullsized":"2024-09-17T23:32:03.050008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:22:15.532367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011995770000000001,"stat_sum":{"num_bytes":61967360,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":20,"num_read":5018,"num_read_kb":38008,"num_write":480,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]},{"pgid":"3.13","version":"159'559","reported_seq":15152,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T23:32:03.140002+0000","last_change":"2024-09-17T23:32:03.051677+0000","last_active":"2024-09-17T23:32:03.140002+0000","last_peered":"2024-09-17T23:32:03.140002+0000","last_clean":"2024-09-17T23:32:03.140002+0000","last_became_active":"2024-09-17T23:31:43.947677+0000","last_became_peered":"2024-09-17T23:31:43.947677+0000","last_unstale":"2024-09-17T23:32:03.140002+0000","last_undegraded":"2024-09-17T23:32:03.140002+0000","last_fullsized":"2024-09-17T23:32:03.140002+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:11.625835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020488139999999999,"stat_sum":{"num_bytes":26830390,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":15,"num_read":15070,"num_read_kb":74825,"num_write":714,"num_write_kb":33700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"10"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":567149587,"num_objects":773,"num_object_clones":189,"num_object_copies":1546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":773,"num_whiteouts":428,"num_read":310667,"num_read_kb":1324704,"num_write":19447,"num_write_kb":480785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7354,"num_bytes_recovered":259965128,"num_keys_recovered":1266,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17711104,"data_stored":136070641,"data_compressed":5716645,"data_compressed_allocated":12849152,"data_compressed_original":133430784,"omap_allocated":7598,"internal_metadata":0},"log_size":18027,"ondisk_log_size":18027,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500684,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45428,"kb_used_data":4572,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94326412,"statfs":{"total":96636764160,"available":96590245888,"internally_reserved":0,"allocated":4681728,"data_stored":23950808,"data_compressed":751667,"data_compressed_allocated":2113536,"data_compressed_original":22789120,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640296,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49804,"kb_used_data":4724,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322036,"statfs":{"total":96636764160,"available":96585764864,"internally_reserved":0,"allocated":4837376,"data_stored":23470442,"data_compressed":864624,"data_compressed_allocated":1970176,"data_compressed_original":22156800,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80968,"kb_used_data":5356,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290872,"statfs":{"total":96636764160,"available":96553852928,"internally_reserved":0,"allocated":5484544,"data_stored":32642717,"data_compressed":1057612,"data_compressed_allocated":2555904,"data_compressed_original":31283712,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":47244640296,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":47244640297,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640296,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68536,"kb_used_data":5204,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94303304,"statfs":{"total":96636764160,"available":96566583296,"internally_reserved":0,"allocated":5328896,"data_stored":29599475,"data_compressed":1036376,"data_compressed_allocated":2551808,"data_compressed_original":28314624,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3403776,"data_stored":28783486,"data_compressed":1036376,"data_compressed_allocated":2551808,"data_compressed_original":28314624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":2250800,"data_compressed":129518,"data_compressed_allocated":262144,"data_compressed_original":2220032,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5304320,"data_stored":34185100,"data_compressed":1986119,"data_compressed_allocated":3895296,"data_compressed_original":33407488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3588096,"data_stored":31845867,"data_compressed":1057612,"data_compressed_allocated":2555904,"data_compressed_original":31283712,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2871296,"data_stored":22069907,"data_compressed":858846,"data_compressed_allocated":1933312,"data_compressed_original":21566976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2224128,"data_stored":16935481,"data_compressed":648174,"data_compressed_allocated":1650688,"data_compressed_original":16637952,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:06.785 INFO:tasks.thrashosds.thrasher:Setting 3.11 to [0, 4] 2024-09-17T23:32:06.785 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.11', '0', '4'] 2024-09-17T23:32:06.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.11 0 4 2024-09-17T23:32:06.785 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:06.777+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:32:06.880 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:06.877+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:06.936 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:06.936 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:06.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:06.990 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:06.988+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:07.065 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:07.065 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:07.065 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:07.066 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:07.066 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:07.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:07.116 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:07.113+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:07.184 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:07.181+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:07.223 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 187.749324097, 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:32:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.89148828399999, 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.890795991, 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.89078441000001, 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:32:07.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:32:07.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.890257163, 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:07.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.89022317999999, 2024-09-17T23:32:07.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:32:07.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.88991258799999, 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:32:07.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:32:07.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:32:07.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.88988645399999, 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:32:07.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:07.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.88938854700001, 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:07.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.266 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.88823882200001, 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:32:07.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:32:07.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:32:07.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.888031224, 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:32:07.271 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:32:07.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.87841532300001, 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:32:07.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.87840508900001, 2024-09-17T23:32:07.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:32:07.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:32:07.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.87839522199999, 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:32:07.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.87838507399999, 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:32:07.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:32:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.87791520600001, 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:32:07.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.87789308999999, 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 133.692219225, 2024-09-17T23:32:07.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:32:07.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:32:07.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:32:07.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.318622597999997, 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:32:07.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:32:07.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:07.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e136)", 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.356035250000001, 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031736553, 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:32:07.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23425 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:07.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864730+0000", 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.485e-06 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864735+0000", 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864747+0000", 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1834999999999999e-05 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864758+0000", 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0431000000000001e-05 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864829+0000", 2024-09-17T23:32:07.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1526000000000006e-05 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864837+0000", 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2509999999999994e-06 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867486+0000", 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002648376 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:07.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867499+0000", 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3553999999999999e-05 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896137+0000", 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.028637427 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896249+0000", 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011187 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896465+0000", 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021632800000000001 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.325 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:07.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:07.328 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:07.325+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:07.428 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:07.425+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:07.476 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:07.476 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:07.476 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:07.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:07.528 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:07.526+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:07.591 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:07.592 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:07.592 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:07.592 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:07.592 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:07.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:07.643 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:07.641+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:07.709 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 147.67098664900001, 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:32:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:32:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:32:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:32:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:32:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 147.67081370899999, 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:32:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:32:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:32:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 147.67077383899999, 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:32:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 147.37626961199999, 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:32:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.12856806299999, 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:32:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 134.118635797, 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:32:07.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:32:07.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:32:07.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:07.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 134.105408748, 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:07.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:32:07.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.564879592, 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:07.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:32:07.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.564858202, 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:32:07.755 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:32:07.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.564841073, 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:32:07.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:32:07.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:32:07.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.56482379099999, 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:32:07.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.564806762, 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:32:07.771 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:32:07.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:32:07.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:32:07.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.564773888, 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:07.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:32:07.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:07.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.564186182, 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:32:07.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:07.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 120.54239404099999, 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.788 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:07.792 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 120.541856853, 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:07.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:32:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 120.54137073699999, 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:32:07.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:07.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.46922813499999, 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:32:07.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:07.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.469206324, 2024-09-17T23:32:07.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:32:07.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.459763454, 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:07.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:07.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:32:07.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:07.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:07.818 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:07.819 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:07.817+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:07.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.11 pg_upmap mapping to [0,4] 2024-09-17T23:32:07.921 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:07.918+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:07.950 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:07.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:08.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:32:08.056 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:08.053+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:08.108 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:08.109 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:08.109 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:08.109 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:08.109 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:08.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:08.124 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:08.122+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:08.136 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:08.137 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:08.154 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:08.154 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:08.170 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:08.170 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:08.187 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:08.187 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:08.204 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:08.221 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:08.221 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:08.225 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:08.225 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:08.225 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:08.222+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:08.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:08.326 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:08.323+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:08.381 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:08.382 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:08.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:08.438 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:08.436+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:08.540 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:08.538+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:08.544 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:08.544 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:08.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:08.640 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:08.638+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:08.660 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:08.660 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:08.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:08.741 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:08.740+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:08.776 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:08.776 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:08.784 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:08.842 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:08.839+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:08.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:32:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:08.916 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:08.957 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:08.954+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:09.044 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:09.041+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:09.073 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.073 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:09.145 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:09.143+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:09.190 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.190 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:09.245 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:09.243+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:09.306 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.306 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:09.356 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:09.353+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:09.447 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:09.444+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:09.463 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.463 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:09.548 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:09.546+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:09.619 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.619 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:09.649 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:09.647+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:09.735 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.735 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:09.786 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:09.784+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:09.851 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.851 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.851 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:09.849+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:09.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:09.865 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:32:09.952 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:09.950+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:09.979 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:09.979 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:09.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:10.053 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:10.050+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:10.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:10.194 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:10.191+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:10.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:32:10.256 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:10.253+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:10.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:10.356 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:10.357 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:10.357 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:10.354+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:10.373 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:10.377 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:10.380 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:10.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:10.389 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:10.406 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:10.406 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:10.423 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:10.424 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:10.440 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:10.493 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:10.491+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:10.493 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:10.494 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:10.494 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:10.494 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:10.494 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:10.503 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:10.557 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:10.556+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:10.612 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:10.612 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:10.612 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:10.612 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.734371146, 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:32:10.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.614 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:32:10.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.616 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.618 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.71874601600001, 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:32:10.621 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.622 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.71870835999999, 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.625 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.626 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:32:10.627 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.628 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.629 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.71869092899999, 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:32:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.632 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:32:10.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.634 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.718655998, 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.636 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:32:10.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:32:10.638 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.639 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.718624578, 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:32:10.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.642 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:32:10.644 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.645 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.71860795200001, 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.647 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:32:10.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.651 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.71858625799999, 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:10.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 163.71857453999999, 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:32:10.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:10.662 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.759436021, 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:32:10.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.666 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:32:10.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.669 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.75548085699999, 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:32:10.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:10.675 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.75544019900001, 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:32:10.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:32:10.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:32:10.680 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.755144552, 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.681 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:10.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:32:10.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:10.686 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.75511225699999, 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:10.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:32:10.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 109.75495744600001, 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:32:10.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:32:10.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.697 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 104.75467112600001, 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.698 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:32:10.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.702 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 104.754602465, 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:32:10.703 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:32:10.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 104.75456049100001, 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:32:10.708 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:32:10.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.754776468000003, 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:32:10.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:32:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:32:10.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:32:10.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:24856 3.b 3:d3034e8c:::rbd_header.1152ea45e50a:head [watch reconnect cookie 140329332260848 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.603457131999999, 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0448234409999999, 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 24856 2024-09-17T23:32:10.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005662+0000", 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9350000000000001e-06 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005669+0000", 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3470000000000004e-06 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005683+0000", 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4419999999999999e-05 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005694+0000", 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0917999999999999e-05 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006157+0000", 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00046328800000000001 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006166+0000", 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:32:10.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041545+0000", 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.035378916 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041630+0000", 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5628000000000001e-05 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045500+0000", 2024-09-17T23:32:10.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038699910000000001 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045553+0000", 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2516000000000003e-05 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.050484+0000", 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049310329999999996 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.728 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:10.729 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:10.729 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:10.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:10.731 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:10.728+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:10.830 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:10.828+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:10.884 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:10.884 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:10.884 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:10.884 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:10.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:10.932 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:10.929+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:11.033 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:11.031+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:11.047 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:11.047 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:11.048 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:11.048 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:11.048 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:11.056 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:11.134 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:11.132+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:11.165 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:11.165 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66863190000001, 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.166 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:32:11.168 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.169 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.170 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66861202000001, 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:32:11.172 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.173 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.174 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66859048200001, 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.176 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.177 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.178 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:11.179 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66669776800001, 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.180 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.181 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:32:11.183 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.666685538, 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.184 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.185 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:32:11.186 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:32:11.187 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:11.188 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66667496900001, 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:32:11.191 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66666508099999, 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.194 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.195 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.66665479900001, 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.197 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:32:11.199 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:32:11.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 153.666638912, 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:32:11.201 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.202 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.203 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.204 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.12771359499999, 2024-09-17T23:32:11.205 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.206 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:32:11.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:32:11.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:32:11.209 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:32:11.210 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 133.125644209, 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.212 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:32:11.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 131.79377684100001, 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:11.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:32:11.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.218 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.218 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.218 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:32:11.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.222 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.308812163, 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:32:11.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.227 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.308779793, 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:32:11.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:32:11.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.30848741600001, 2024-09-17T23:32:11.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:32:11.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.302663278, 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:32:11.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.243 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.30262177500001, 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.248 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.259883508, 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.253 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.259781167, 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:11.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:32:11.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:32:11.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 32.279467265000001, 2024-09-17T23:32:11.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:11.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:32:11.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.264 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:11.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:11.266 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:11.264+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:11.367 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:11.365+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:11.374 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:11.374 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:11.374 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:11.374 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:11.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:11.468 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:11.466+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:11.489 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:11.489 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:11.489 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:11.490 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:11.490 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:11.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:11.583 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:11.581+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.5594.0:1626 3.1e e112/107 3:78a74853:::rbd_data.1152d7c8a97e.000000000000000e:head v 112'474, mlcod=109'472)", 2024-09-17T23:32:11.648 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.264672423, 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057994730000000003, 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.649 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380067+0000", 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380068+0000", 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5900000000000003e-07 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380072+0000", 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9899999999999999e-06 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.650 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380079+0000", 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4000000000000003e-06 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380084+0000", 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9359999999999998e-06 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380132+0000", 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7608999999999998e-05 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.651 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.380150+0000", 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8754e-05 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385832+0000", 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0056812099999999999 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.385867+0000", 2024-09-17T23:32:11.652 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5215000000000001e-05 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21425 3.17 3:e879067b:::rbd_data.1152d7c8a97e.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 54.662421371000001, 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055231200000000003, 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.653 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21425 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982318+0000", 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982319+0000", 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9900000000000009e-07 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982325+0000", 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6400000000000002e-06 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982339+0000", 2024-09-17T23:32:11.655 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3688e-05 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982346+0000", 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4939999999999997e-06 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982405+0000", 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8724999999999998e-05 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.656 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982478+0000", 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2955999999999997e-05 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.982575+0000", 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6926e-05 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.984850+0000", 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022756220000000001 2024-09-17T23:32:11.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987782+0000", 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002931917 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987817+0000", 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4356000000000001e-05 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:16.987841+0000", 2024-09-17T23:32:11.658 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4797e-05 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21650 3.0 3:02844c61:::rbd_header.11525992f601:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114)", 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 53.561788733999997, 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0092824489999999999, 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.659 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21650 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082951+0000", 2024-09-17T23:32:11.660 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082952+0000", 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.398e-06 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082962+0000", 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8470000000000008e-06 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.661 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082974+0000", 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.185e-05 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.082985+0000", 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1250000000000001e-05 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083048+0000", 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2340000000000003e-05 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.662 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.083052+0000", 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3019999999999996e-06 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.086966+0000", 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0039144440000000004 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087069+0000", 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00010230699999999999 2024-09-17T23:32:11.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 3", 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.087449+0000", 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038016500000000002 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.088255+0000", 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00080638400000000005 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.664 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092155+0000", 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038993320000000001 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092189+0000", 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4786000000000002e-05 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:18.092233+0000", 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4044e-05 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22097 3.1f 3:fb0bd643:::rbd_data.1152c10d4b9c.0000000000000020:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.409261088000001, 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053145850000000001, 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22097 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235479+0000", 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.667 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235480+0000", 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4900000000000004e-07 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235485+0000", 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.147e-06 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.669 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235497+0000", 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.207e-05 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235505+0000", 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.371e-06 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235578+0000", 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2852e-05 2024-09-17T23:32:11.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235723+0000", 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014481499999999999 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235905+0000", 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000182311 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.237178+0000", 2024-09-17T23:32:11.671 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001272942 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240547+0000", 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0033686039999999999 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240605+0000", 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8375000000000003e-05 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.672 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240793+0000", 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000188149 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22099 3.17 3:e89226ba:::rbd_data.1152c10d4b9c.0000000000000022:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.409123848999997, 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053126650000000003, 2024-09-17T23:32:11.673 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22099 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235616+0000", 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235617+0000", 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2699999999999998e-07 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235622+0000", 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5470000000000004e-06 2024-09-17T23:32:11.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235634+0000", 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1983e-05 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235650+0000", 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5192e-05 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.676 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235707+0000", 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7241000000000003e-05 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235785+0000", 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8572000000000006e-05 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.235903+0000", 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000117166 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.677 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240794+0000", 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0048912570000000004 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240822+0000", 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8606e-05 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240875+0000", 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2247000000000003e-05 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.678 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240929+0000", 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3927000000000001e-05 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22107 3.17 3:ec66b126:::rbd_data.1152c10d4b9c.0000000000000036:head [delete] snapc 2a=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 49.403966689000001, 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053327360000000002, 2024-09-17T23:32:11.679 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22107 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.680 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240773+0000", 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240774+0000", 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.361e-06 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240782+0000", 2024-09-17T23:32:11.681 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9240000000000007e-06 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240795+0000", 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2425e-05 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240803+0000", 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4759999999999998e-06 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.682 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.240962+0000", 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000159071 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241059+0000", 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6650999999999993e-05 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.241201+0000", 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000141977 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.683 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.243180+0000", 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001978778 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246016+0000", 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0028363020000000002 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246067+0000", 2024-09-17T23:32:11.684 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0494999999999997e-05 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.246106+0000", 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9276000000000001e-05 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22907 3.0 3:07f1f9a0:::rbd_data.11524e94aa61.0000000000000007:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:32:11.685 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 40.668912704, 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053158520000000003, 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22907 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:32:11.686 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975827+0000", 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975829+0000", 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0490000000000002e-06 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.687 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975838+0000", 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.3260000000000002e-06 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975859+0000", 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0378999999999999e-05 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.688 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975870+0000", 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1488000000000001e-05 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.975948+0000", 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8199000000000001e-05 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976110+0000", 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000161977 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.689 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.976234+0000", 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012316599999999999 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.977277+0000", 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0010435679999999999 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981028+0000", 2024-09-17T23:32:11.690 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037512130000000002 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981092+0000", 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3468999999999998e-05 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:30.981143+0000", 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1017999999999998e-05 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.691 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:22969 3.17 3:e879c454:::rbd_header.1152324fa224:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 39.755743262999999, 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058282270000000001, 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 22969 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888996+0000", 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.693 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.888998+0000", 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.11e-06 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889005+0000", 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9500000000000004e-06 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889017+0000", 2024-09-17T23:32:11.694 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2478e-05 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889029+0000", 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.168e-05 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889311+0000", 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028192400000000002 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.695 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889435+0000", 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000124046 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.889886+0000", 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00045141000000000002 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.892147+0000", 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.002260813 2024-09-17T23:32:11.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894710+0000", 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025627710000000001 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894768+0000", 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8059000000000001e-05 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.697 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:31.894825+0000", 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6986e-05 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:23135 3.1b e131/130 3:d93fe334:::rbd_object_map.11527998816:head v 131'391, mlcod=0'0)", 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 36.541531397, 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053043279999999996, 2024-09-17T23:32:11.698 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103208+0000", 2024-09-17T23:32:11.699 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103209+0000", 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4200000000000002e-07 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103215+0000", 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0399999999999998e-06 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.700 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103222+0000", 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4919999999999996e-06 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103231+0000", 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6049999999999998e-06 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103277+0000", 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6641999999999998e-05 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.103302+0000", 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4601e-05 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108442+0000", 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0051404129999999999 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:35.108513+0000", 2024-09-17T23:32:11.702 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0192999999999997e-05 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24442 3.11 e145/138 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:head v 145'346, mlcod=140'344)", 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 22.569087124999999, 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054889680000000003, 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.703 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.704 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075653+0000", 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7499999999999999e-07 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075660+0000", 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3430000000000001e-06 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075673+0000", 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2758000000000001e-05 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075680+0000", 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0190000000000001e-06 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075731+0000", 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1872000000000001e-05 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.075755+0000", 2024-09-17T23:32:11.706 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3668000000000001e-05 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081093+0000", 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053383019999999996 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:49.081142+0000", 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8231e-05 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.707 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:24961 3.1a e150/139 3:5d9a7141:::rbd_data.1152ea45e50a.0000000000000015:head v 150'858, mlcod=146'854)", 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 17.459354906000002, 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0055120910000000002, 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.708 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185385+0000", 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4499999999999997e-07 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.709 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185391+0000", 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3820000000000003e-06 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185401+0000", 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0271999999999999e-05 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185408+0000", 2024-09-17T23:32:11.710 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8859999999999998e-06 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185462+0000", 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4234000000000003e-05 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.185483+0000", 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0747e-05 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.711 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.190854+0000", 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053711899999999996 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.190897+0000", 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.712 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25112 3.17 3:e827691a:::rbd_data.1152ce7c866.0000000000000001:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.693511044999999, 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069415060000000001, 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25112 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.713 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951229+0000", 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.714 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951232+0000", 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1700000000000001e-06 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951250+0000", 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7629999999999999e-05 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.715 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951257+0000", 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.401e-06 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952436+0000", 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0011780849999999999 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952728+0000", 2024-09-17T23:32:11.716 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000292379 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952856+0000", 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012781000000000001 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.954326+0000", 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0014700410000000001 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.717 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958040+0000", 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037147080000000002 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958100+0000", 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9796000000000001e-05 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958170+0000", 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9950000000000006e-05 2024-09-17T23:32:11.718 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25113 3.17 3:ea59ff92:::rbd_data.1152ce7c866.0000000000000002:head [delete] snapc 17=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.693404735, 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0069436059999999997, 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.719 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25113 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.720 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951335+0000", 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951336+0000", 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2099999999999996e-07 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951339+0000", 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.3560000000000001e-06 2024-09-17T23:32:11.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951351+0000", 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2422999999999999e-05 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.951359+0000", 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.131e-06 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.722 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953211+0000", 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018522359999999999 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953338+0000", 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012692899999999999 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.953425+0000", 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7439000000000001e-05 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.955541+0000", 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021159479999999999 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958181+0000", 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0026395770000000002 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958233+0000", 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2537999999999997e-05 2024-09-17T23:32:11.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958279+0000", 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5408000000000003e-05 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25115 3.1e e153/138 3:7ebe4efd:::rbd_data.1152ce7c866.0000000000000004:head v 153'526, mlcod=146'524)", 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.692573063999999, 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0059104259999999999, 2024-09-17T23:32:11.725 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:11.726 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952167+0000", 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6100000000000002e-07 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952171+0000", 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8770000000000003e-06 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.727 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952178+0000", 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6579999999999996e-06 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952183+0000", 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5180000000000004e-06 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952263+0000", 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0242999999999997e-05 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.728 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952283+0000", 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9678999999999999e-05 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958014+0000", 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0057307460000000001 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958077+0000", 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3343999999999995e-05 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.729 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25119 3.11 e153/138 3:8916a254:::rbd_data.1152ce7c866.0000000000000032:head v 153'351, mlcod=146'348)", 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 14.692543756999999, 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0058297339999999996, 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.730 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952196+0000", 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1199999999999999e-07 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.731 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952204+0000", 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.0290000000000005e-06 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952211+0000", 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7599999999999997e-06 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952216+0000", 2024-09-17T23:32:11.732 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0069999999999997e-06 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952606+0000", 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00038973799999999999 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.952625+0000", 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9636999999999999e-05 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.733 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.957982+0000", 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0053570220000000003 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:56.958026+0000", 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3229000000000002e-05 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:25192 3.1a e155/139 3:5d1d36b7:::rbd_object_map.11526e6e10a0:head v 155'861, mlcod=154'859)", 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:58.873578+0000", 2024-09-17T23:32:11.734 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 12.771161381000001, 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0073394310000000004, 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873578+0000", 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873578+0000", 2024-09-17T23:32:11.735 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873579+0000", 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3999999999999998e-07 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873584+0000", 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3299999999999998e-06 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.736 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873601+0000", 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.6714999999999998e-05 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873610+0000", 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7680000000000003e-06 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873669+0000", 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8576999999999998e-05 2024-09-17T23:32:11.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.873686+0000", 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7280000000000001e-05 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.880870+0000", 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0071837300000000002 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:58.880918+0000", 2024-09-17T23:32:11.738 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7790999999999997e-05 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:25726 3.17 3:edcf7f41:::rbd_header.115262cdab6:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e158)", 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:32:02.041894+0000", 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 9.6028454379999992, 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054082499999999999, 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:11.739 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 25726 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041894+0000", 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041894+0000", 2024-09-17T23:32:11.740 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041896+0000", 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8899999999999999e-06 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041905+0000", 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3080000000000008e-06 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.741 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041922+0000", 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7774e-05 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.041932+0000", 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7690000000000008e-06 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.042010+0000", 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7461999999999998e-05 2024-09-17T23:32:11.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.042018+0000", 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3080000000000008e-06 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.044428+0000", 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024098639999999998 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.743 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.044554+0000", 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000126155 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.045062+0000", 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00050860800000000002 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.046499+0000", 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001436547 2024-09-17T23:32:11.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.047183+0000", 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00068358600000000002 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.047249+0000", 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6548e-05 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:02.047303+0000", 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3430999999999997e-05 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(osd.0.0:6464 3.12 e163/133 3:4cc754a9:::rbd_data.1152ddb99355.000000000000000e:e v 163'432, mlcod=161'429)", 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:32:06.866841+0000", 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.7778986950000002, 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0054632880000000002, 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.866841+0000", 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.866841+0000", 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.866842+0000", 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2600000000000001e-07 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.866851+0000", 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6279999999999994e-06 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.866863+0000", 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2427e-05 2024-09-17T23:32:11.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.866873+0000", 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8120000000000002e-06 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.868453+0000", 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001580363 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.749 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.868477+0000", 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4029999999999999e-05 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.872255+0000", 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0037778439999999998 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.872304+0000", 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9258000000000002e-05 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.750 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(osd.3.0:5026 3.16 e163/151 3:6cbca22a:::rbd_data.1152ddb99355.0000000000000001:e v 163'360, mlcod=161'357)", 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:32:06.871467+0000", 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 4.7732729870000004, 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0062687150000000002, 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871467+0000", 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871467+0000", 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871468+0000", 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1900000000000001e-07 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871476+0000", 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6929999999999996e-06 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871492+0000", 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5160000000000001e-05 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871501+0000", 2024-09-17T23:32:11.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.5689999999999994e-06 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871564+0000", 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2898000000000007e-05 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.871587+0000", 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3031999999999999e-05 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.877679+0000", 2024-09-17T23:32:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0060923410000000002 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:06.877735+0000", 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6103000000000001e-05 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(osd.0.0:6504 3.e e164/152 3:741d178d:::rbd_data.1152fa98c8a5.0000000000000006:d v 164'743, mlcod=162'737)", 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:32:07.878474+0000", 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 3.7662654789999999, 2024-09-17T23:32:11.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0063020130000000004, 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878474+0000", 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878474+0000", 2024-09-17T23:32:11.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878475+0000", 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8899999999999998e-07 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878490+0000", 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5347999999999999e-05 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878501+0000", 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0855e-05 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878506+0000", 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1699999999999996e-06 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878543+0000", 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6693999999999998e-05 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.878558+0000", 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4834000000000001e-05 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.884745+0000", 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0061871360000000002 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:07.884776+0000", 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1186999999999998e-05 2024-09-17T23:32:11.759 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.760 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.760 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:11.760 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:11.760 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:11.761 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:11.759+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:11.762 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:11.862 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:11.860+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:11.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:32:11.891 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:11.892 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:11.892 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:11.892 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:11.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:11.988 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:11.986+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:12.064 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:12.062+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:12.078 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:12.079 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:12.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:12.165 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:12.163+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:12.237 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.392903507, 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.014080141, 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:32:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841739+0000", 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.841740+0000", 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4499999999999997e-07 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846411+0000", 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046718610000000002 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846423+0000", 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1800000000000001e-05 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846433+0000", 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4630000000000007e-06 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846794+0000", 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036171599999999999 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.846888+0000", 2024-09-17T23:32:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3236e-05 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.847129+0000", 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024083899999999999 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852310+0000", 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051818300000000001 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.852327+0000", 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6161e-05 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855807+0000", 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034802469999999 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855819+0000", 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2442999999999999e-05 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3.1feaf7fd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.383331408, 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004859194, 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:32:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851311+0000", 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.851312+0000", 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.155e-06 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853648+0000", 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335227 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853658+0000", 2024-09-17T23:32:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0525e-05 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853666+0000", 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0879999999999995e-06 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853724+0000", 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8066000000000003e-05 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.853735+0000", 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0329000000000001e-05 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856161+0000", 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0024265080000001 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856170+0000", 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2960000000000004e-06 2024-09-17T23:32:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3.ecf802a3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.379800902, 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99989275899999996, 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854842+0000", 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4200000000000001e-07 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854846+0000", 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4860000000000002e-06 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.854853+0000", 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855106+0000", 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000253188 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855240+0000", 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000133824 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.855251+0000", 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1318000000000001e-05 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854721+0000", 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99946962100000003 2024-09-17T23:32:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.854734+0000", 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3586e-05 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.378306196, 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99986823400000002, 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:32:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856336+0000", 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856337+0000", 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-07 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856341+0000", 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999999e-06 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856352+0000", 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1028e-05 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856360+0000", 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0469999999999994e-06 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856416+0000", 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6059e-05 2024-09-17T23:32:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856426+0000", 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2930000000000006e-06 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856196+0000", 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99976998100000003 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856205+0000", 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9400000000000008e-06 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.378123277, 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99971795399999996, 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856519+0000", 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856520+0000", 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856524+0000", 2024-09-17T23:32:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5829999999999998e-06 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856535+0000", 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1090999999999999e-05 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856542+0000", 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3080000000000002e-06 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856871+0000", 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032897200000000002 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856879+0000", 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3639999999999999e-06 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856228+0000", 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99934879899999995 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856237+0000", 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.166e-06 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:520 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.378009776, 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920056999999995, 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 520 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856633+0000", 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5900000000000002e-07 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856636+0000", 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.587e-06 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856644+0000", 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8320000000000006e-06 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856649+0000", 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2780000000000001e-06 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856750+0000", 2024-09-17T23:32:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101498 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856759+0000", 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855663+0000", 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99890390500000004 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855833+0000", 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017052699999999999 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:521 3.3 3.d27c1d23 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.377985669, 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99925141299999998, 2024-09-17T23:32:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 521 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856657+0000", 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856658+0000", 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2300000000000001e-07 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856660+0000", 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856668+0000", 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8199999999999997e-06 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856673+0000", 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856787+0000", 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113367 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856793+0000", 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-06 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855900+0000", 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99910724500000003 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855908+0000", 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5080000000000005e-06 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:532 3.3 3.722a5723 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.377911338, 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99920949299999995, 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 532 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856731+0000", 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856732+0000", 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856734+0000", 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6699999999999998e-06 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856742+0000", 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3590000000000002e-06 2024-09-17T23:32:12.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856747+0000", 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9010000000000002e-06 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856815+0000", 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8349999999999994e-05 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856821+0000", 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9320000000000001e-06 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855932+0000", 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999111372 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855941+0000", 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3720000000000005e-06 2024-09-17T23:32:12.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:533 3.14 3.43e68814 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.377887789, 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001720326, 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 533 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:12.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856755+0000", 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7700000000000005e-07 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856758+0000", 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5620000000000002e-06 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856766+0000", 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7540000000000006e-06 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.856771+0000", 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0730000000000004e-06 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858676+0000", 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001904974 2024-09-17T23:32:12.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.858683+0000", 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6550000000000004e-06 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858467+0000", 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.999783266 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858475+0000", 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5650000000000002e-06 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3.ad76901d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.374716588, 2024-09-17T23:32:12.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99634244199999999, 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:12.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859926+0000", 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9200000000000001e-07 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859929+0000", 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5380000000000001e-06 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859935+0000", 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3249999999999996e-06 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.859942+0000", 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6279999999999999e-06 2024-09-17T23:32:12.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4525999999999997e-05 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.860037+0000", 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.026e-05 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856261+0000", 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.99622437900000005 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856268+0000", 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:496 3.1d 3:ba97d4c3:::rbd_data.115240362710.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.370308252, 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.992031989, 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 496 2024-09-17T23:32:12.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864334+0000", 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.864335+0000", 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2999999999999999e-07 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881357+0000", 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017021967999999998 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881369+0000", 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1574e-05 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881374+0000", 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2009999999999998e-06 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881437+0000", 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3118000000000001e-05 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.881448+0000", 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0512e-05 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856293+0000", 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97484585999999995 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856366+0000", 2024-09-17T23:32:12.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2925999999999999e-05 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:498 3.1d 3:bfef57f8:::rbd_data.115240362710.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.343036145, 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97403922499999995, 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:12.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 498 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891606+0000", 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.891607+0000", 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4600000000000005e-07 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898966+0000", 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073587219999999998 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898978+0000", 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.898983+0000", 2024-09-17T23:32:12.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4609999999999999e-06 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899039+0000", 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5336e-05 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.899048+0000", 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856398+0000", 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.957350068 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.857501+0000", 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011029480000000001 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.858096+0000", 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00059545699999999997 2024-09-17T23:32:12.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863194+0000", 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050974480000000001 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865529+0000", 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002335335 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:12.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865597+0000", 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8440000000000002e-05 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.865646+0000", 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8257000000000002e-05 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.307 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:505 3.3 3:c5401f37:::rbd_data.115240362710.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33385399300001, 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96043457700000001, 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 505 2024-09-17T23:32:12.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900789+0000", 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0499999999999999e-07 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900792+0000", 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3890000000000001e-06 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900802+0000", 2024-09-17T23:32:12.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2739999999999996e-06 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900807+0000", 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6409999999999999e-06 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900865+0000", 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8146999999999997e-05 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900874+0000", 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.855985+0000", 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95511061200000003 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856129+0000", 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014377500000000001 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.856352+0000", 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022378799999999999 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:12.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.860878+0000", 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045257500000000003 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861128+0000", 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025022599999999999 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861185+0000", 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861223+0000", 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8111000000000002e-05 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:512 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.3338225, 2024-09-17T23:32:12.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96164719700000001, 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 512 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900820+0000", 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900821+0000", 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6000000000000004e-07 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900823+0000", 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.593e-06 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900830+0000", 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6189999999999996e-06 2024-09-17T23:32:12.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900835+0000", 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2800000000000003e-06 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900886+0000", 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0392000000000001e-05 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900892+0000", 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6200000000000001e-06 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862231+0000", 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96133849599999999 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862361+0000", 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013059599999999999 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862467+0000", 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106041 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.321 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:513 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 4030464~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.333796808, 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96186341200000003, 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 513 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900846+0000", 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.89e-07 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900849+0000", 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5799999999999999e-06 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900855+0000", 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3690000000000003e-06 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900860+0000", 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9819999999999999e-06 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900915+0000", 2024-09-17T23:32:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4882000000000001e-05 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900921+0000", 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862529+0000", 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96160778400000002 2024-09-17T23:32:12.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862610+0000", 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1547999999999999e-05 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862709+0000", 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8907999999999996e-05 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:541 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 1933312~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33365225599999, 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96190392499999999, 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 541 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900990+0000", 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900991+0000", 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900993+0000", 2024-09-17T23:32:12.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1909999999999999e-06 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.900999+0000", 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901004+0000", 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9490000000000004e-06 2024-09-17T23:32:12.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901040+0000", 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6056999999999997e-05 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.901047+0000", 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9829999999999999e-06 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862738+0000", 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96169125600000005 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862808+0000", 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9511999999999994e-05 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862894+0000", 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6464000000000004e-05 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:566 3.1d 3:b8096eb5:::rbd_data.1152adcf8b1e.0000000000000003:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.332084139, 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96062620799999998, 2024-09-17T23:32:12.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 566 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902558+0000", 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902559+0000", 2024-09-17T23:32:12.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902565+0000", 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902571+0000", 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9340000000000003e-06 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902575+0000", 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902619+0000", 2024-09-17T23:32:12.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4237999999999999e-05 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902630+0000", 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0583e-05 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.862923+0000", 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.96029314399999999 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863000+0000", 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6680000000000004e-05 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.863185+0000", 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018507699999999999 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.340 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:580 3.3 3:c4ea544e:::rbd_data.1152adcf8b1e.0000000000000012:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33188679200001, 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95870249399999996, 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 580 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902756+0000", 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902759+0000", 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-06 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902766+0000", 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2150000000000004e-06 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902772+0000", 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902815+0000", 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3443999999999999e-05 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902822+0000", 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9659999999999999e-06 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861267+0000", 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95844459500000001 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861356+0000", 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8976999999999999e-05 2024-09-17T23:32:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861458+0000", 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102678 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:581 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 163840~1536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.33186047700001, 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95890952799999996, 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 581 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902782+0000", 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902783+0000", 2024-09-17T23:32:12.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7999999999999995e-07 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902785+0000", 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902794+0000", 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5180000000000004e-06 2024-09-17T23:32:12.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902799+0000", 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4009999999999996e-06 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902839+0000", 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0145000000000003e-05 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902845+0000", 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2040000000000002e-06 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859256+0000", 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95641094900000001 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.859336+0000", 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9276000000000005e-05 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861692+0000", 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023560209999999998 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5325.0:582 3.14 3:281167c2:::rbd_data.1152adcf8b1e.0000000000000013:1b [sparse-read 2457600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 111.331832134, 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95917654399999996, 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5325", 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 582 2024-09-17T23:32:12.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902810+0000", 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902811+0000", 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902813+0000", 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3319999999999999e-06 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:12.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902820+0000", 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6259999999999997e-06 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902825+0000", 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902863+0000", 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7669000000000001e-05 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:20.902868+0000", 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1209999999999998e-06 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861729+0000", 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95886096499999995 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861793+0000", 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3929000000000004e-05 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:21.861987+0000", 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019417499999999999 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:12.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:12.360 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:12.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:12.362 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:12.359+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:12.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:32:12.470 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:12.470 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:12.470 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:12.470 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:12.470 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:12.468+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:12.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:12.562 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:12.560+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:12.577 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:12.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:12.595 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:12.611 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:12.611 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:12.629 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:12.629 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:12.629 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:12.629 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:12.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:12.646 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:12.646 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:12.663 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:12.661+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:12.664 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:12.664 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:12.746 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:12.746 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:12.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:12.765 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:12.763+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:12.863 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:12.863 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:12.865 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:12.863+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:12.867 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:32:12.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:12.881 INFO:tasks.thrashosds.thrasher:in_osds: [0, 3, 4, 5] out_osds: [2, 1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T23:32:12.881 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T23:32:12.881 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-17T23:32:12.881 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T23:32:12.881 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T23:32:12.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-17T23:32:12.966 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:12.964+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:12.985 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:12.985 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:12.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:13.034 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T23:32:13.034 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:13.084 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:13.082+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:13.104 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.104 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:13.168 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:13.166+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:13.221 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.221 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.230 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:13.274 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:13.272+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:13.337 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.337 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:13.387 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:13.385+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:13.470 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:13.468+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:13.493 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.494 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:13.571 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:13.569+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.609 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:13.672 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:13.670+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:13.725 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.725 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.734 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:13.779 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:13.777+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:13.842 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:13.842 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:13.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:13.892 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:13.890+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:13.975 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:13.973+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:14.009 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:14.009 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:14.018 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:14.075 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:14.073+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:14.125 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:14.126 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:14.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:14.177 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:14.175+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:14.277 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:14.275+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:14.281 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:14.281 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:14.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:14.378 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:14.377+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:14.411 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:14.411 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:14.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:14.506 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:14.505+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:14.572 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:14.572 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:14.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:14.581 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:14.578+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:14.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T23:32:14.681 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:14.679+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:14.730 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:14.730 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:14.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:14.781 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:14.779+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:14.817 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:14.817 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:14.835 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:14.835 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:14.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:14.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:14.860 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:14.860 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:14.870 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:14.870 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:14.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:14.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T23:32:14.892 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:14.893 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:14.893 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:14.890+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:14.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:14.984 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:14.984 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:14.985 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:14.982+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:14.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:15.085 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:15.083+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:15.105 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4192.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e13)", 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 195.631002822, 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.21207129799999999, 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4192", 2024-09-17T23:32:15.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.37:0/2341924081", 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471440+0000", 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471442+0000", 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.125e-06 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471450+0000", 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2160000000000006e-06 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471465+0000", 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4983999999999999e-05 2024-09-17T23:32:15.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471478+0000", 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3538e-05 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471586+0000", 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010805400000000001 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.471595+0000", 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5730000000000008e-06 2024-09-17T23:32:15.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474062+0000", 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024670759999999999 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.474071+0000", 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5590000000000006e-06 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680514+0000", 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.20644351799999999 2024-09-17T23:32:15.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680721+0000", 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020689600000000001 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.680898+0000", 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000176434 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.681911+0000", 2024-09-17T23:32:15.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010137360000000001 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683441+0000", 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015294799999999999 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683476+0000", 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5623000000000001e-05 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:28:59.683511+0000", 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4486000000000002e-05 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11920 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 3702784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.77316700899999, 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.051046531999999999, 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11920 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329275+0000", 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329946+0000", 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00067014799999999997 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329948+0000", 2024-09-17T23:32:15.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8899999999999999e-06 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329955+0000", 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9589999999999998e-06 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329960+0000", 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6609999999999999e-06 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380153+0000", 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050192987000000001 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380245+0000", 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1708999999999999e-05 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380322+0000", 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7177999999999999e-05 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11931 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.772474716, 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050561202999999999, 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11931 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329968+0000", 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7599999999999998e-07 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329969+0000", 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.513e-06 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329973+0000", 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.67e-06 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329976+0000", 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380366+0000", 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050390496 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380450+0000", 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4012999999999997e-05 2024-09-17T23:32:15.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380529+0000", 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8578999999999994e-05 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11938 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4071424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.77246313500001, 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050766024, 2024-09-17T23:32:15.122 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11938 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.329979+0000", 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330443+0000", 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046343099999999999 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330447+0000", 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330458+0000", 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0883000000000001e-05 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330465+0000", 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2629999999999997e-06 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380577+0000", 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050112085000000001 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380665+0000", 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8047000000000006e-05 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380745+0000", 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9913000000000006e-05 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11952 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.771935888, 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050623168000000003, 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11952 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330506+0000", 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330507+0000", 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9900000000000001e-07 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330510+0000", 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0910000000000001e-06 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330526+0000", 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5543e-05 2024-09-17T23:32:15.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330531+0000", 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6450000000000002e-06 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.380963+0000", 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050431401000000001 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381049+0000", 2024-09-17T23:32:15.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6718999999999994e-05 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381130+0000", 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0270000000000002e-05 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11958 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4096000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.77190190499999, 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050788273000000002, 2024-09-17T23:32:15.131 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11958 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330540+0000", 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330815+0000", 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0002742 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330820+0000", 2024-09-17T23:32:15.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330832+0000", 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2135e-05 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330839+0000", 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3640000000000001e-06 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381165+0000", 2024-09-17T23:32:15.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050326088999999997 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381250+0000", 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5232e-05 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381329+0000", 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8356999999999995e-05 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11967 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.77159131299999, 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050733356, 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11967 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330851+0000", 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330852+0000", 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330854+0000", 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330862+0000", 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9500000000000001e-06 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330868+0000", 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5339999999999999e-06 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381399+0000", 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050530737999999999 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381493+0000", 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4098000000000006e-05 2024-09-17T23:32:15.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381584+0000", 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1766000000000002e-05 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11974 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4112384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.77156517899999, 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050929960000000003, 2024-09-17T23:32:15.140 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11974 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:15.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.330877+0000", 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331339+0000", 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046151900000000002 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331344+0000", 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9429999999999999e-06 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331355+0000", 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331363+0000", 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6219999999999998e-06 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381636+0000", 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050272569000000003 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381723+0000", 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7563000000000002e-05 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381807+0000", 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3825000000000005e-05 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:11982 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 4120576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.77106727200001, 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050653457999999998, 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11982 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331375+0000", 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331376+0000", 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3600000000000004e-07 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331378+0000", 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6259999999999999e-06 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331395+0000", 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6986999999999998e-05 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.331402+0000", 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7129999999999999e-06 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381857+0000", 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050455439999999997 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.381947+0000", 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9680999999999997e-05 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382029+0000", 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1475000000000002e-05 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12010 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4440064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.76991754700001, 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050528688000000002, 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12010 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332525+0000", 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332703+0000", 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000178118 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332707+0000", 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9509999999999999e-06 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332715+0000", 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7670000000000003e-06 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332721+0000", 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5529999999999998e-06 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382893+0000", 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050172091000000002 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.382974+0000", 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0761999999999997e-05 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383054+0000", 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9445999999999998e-05 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12015 3.d 3:b6c8bf64:::rbd_data.11521dab6f51.0000000000000008:11 [read 4448256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.769709949, 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050515721, 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12015 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.332732+0000", 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333606+0000", 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087377600000000004 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333612+0000", 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6030000000000004e-06 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333624+0000", 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2447e-05 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.333633+0000", 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3010000000000007e-06 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383089+0000", 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049456587000000003 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383169+0000", 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9375999999999994e-05 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.383248+0000", 2024-09-17T23:32:15.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9630999999999997e-05 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12300 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6995968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.76009404800001, 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050533000000000002, 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12300 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342348+0000", 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342349+0000", 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342350+0000", 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1909999999999999e-06 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342353+0000", 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3270000000000001e-06 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342356+0000", 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392718+0000", 2024-09-17T23:32:15.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050362614999999999 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392805+0000", 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7003999999999996e-05 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392881+0000", 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6174000000000006e-05 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12305 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 7004160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.76008381400001, 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050658688, 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12305 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342359+0000", 2024-09-17T23:32:15.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.04e-07 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342360+0000", 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342363+0000", 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342365+0000", 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5399999999999998e-06 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392913+0000", 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050547891999999997 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.392973+0000", 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9586999999999997e-05 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393017+0000", 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4356e-05 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12315 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6946816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.760073947, 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050757082000000002, 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12315 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342368+0000", 2024-09-17T23:32:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342369+0000", 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342370+0000", 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2869999999999999e-06 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342373+0000", 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0690000000000002e-06 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342376+0000", 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5950000000000001e-06 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393044+0000", 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050668707 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393084+0000", 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0077999999999998e-05 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393126+0000", 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1139e-05 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12322 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6955008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.76006379899999, 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.05085638, 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12322 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342379+0000", 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342819+0000", 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044005800000000001 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342823+0000", 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0810000000000004e-06 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342831+0000", 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4109999999999996e-06 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342837+0000", 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1430000000000003e-06 2024-09-17T23:32:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393152+0000", 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050314390000000001 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393191+0000", 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9660999999999999e-05 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393235+0000", 2024-09-17T23:32:15.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3636e-05 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12327 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.75959393100001, 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050507415999999999, 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12327 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342848+0000", 2024-09-17T23:32:15.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342849+0000", 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0299999999999999e-07 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342851+0000", 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4590000000000001e-06 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342858+0000", 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8040000000000004e-06 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342863+0000", 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393260+0000", 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050396654999999999 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393298+0000", 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8464999999999998e-05 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393356+0000", 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7385e-05 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12332 3.d 3:b39d15c1:::rbd_data.11521dab6f51.0000000000000002:11 [read 6971392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.75957181499999, 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050589947000000003, 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12332 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:15.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.342871+0000", 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343128+0000", 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025781399999999998 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343132+0000", 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0509999999999998e-06 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343141+0000", 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6470000000000004e-06 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.343147+0000", 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393381+0000", 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.050233291999999999 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393420+0000", 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9137000000000002e-05 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.393461+0000", 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0734e-05 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4968.0:12635 3.14 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.57389795, 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.085928648999999996, 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4968", 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2381057459", 2024-09-17T23:32:15.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 12635 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528544+0000", 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.528549+0000", 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3830000000000002e-06 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586376+0000", 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057826808 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586403+0000", 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7662999999999999e-05 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586414+0000", 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0574000000000001e-05 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586476+0000", 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2242999999999993e-05 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.586597+0000", 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121005 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.587899+0000", 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013021510000000001 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.602348+0000", 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014449071000000001 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614351+0000", 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012003026999999999 2024-09-17T23:32:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614393+0000", 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614473+0000", 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0084000000000001e-05 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:22305 1.0 1:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 49.200301322999998, 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95904951900000002, 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22305 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902141+0000", 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902142+0000", 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8599999999999996e-07 2024-09-17T23:32:15.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902147+0000", 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8210000000000001e-06 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902158+0000", 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0742e-05 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902165+0000", 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2409999999999998e-06 2024-09-17T23:32:15.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902225+0000", 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0399999999999998e-05 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:25.902237+0000", 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1287999999999999e-05 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861045+0000", 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95880856699999994 2024-09-17T23:32:15.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:26.861191+0000", 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014547400000000001 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e136)", 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 35.237713974999998, 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.031736553, 2024-09-17T23:32:15.196 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4434", 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23425 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864728+0000", 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864730+0000", 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.485e-06 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864735+0000", 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-09-17T23:32:15.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864747+0000", 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1834999999999999e-05 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864758+0000", 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0431000000000001e-05 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864829+0000", 2024-09-17T23:32:15.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1526000000000006e-05 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.864837+0000", 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2509999999999994e-06 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867486+0000", 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002648376 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-17T23:32:15.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:39.867499+0000", 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3553999999999999e-05 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896137+0000", 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.028637427 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896249+0000", 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011187 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:31:40.896465+0000", 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021632800000000001 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.202 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:15.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:15.204 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:15.201+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:15.304 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:15.303+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:15.351 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-17T23:32:15.352 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:15.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:15.446 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:15.444+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:15.506 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:15.504+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:15.509 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:15.509 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-17T23:32:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-17T23:32:15.510 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-17T23:32:15.510 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:15.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:15.607 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:15.605+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:15.668 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14821 3.11 3:8a3f55ce:::rbd_data.1152c1142907.0000000000000048:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 155.63007482099999, 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012370493999999999, 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14821 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035656+0000", 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035657+0000", 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1800000000000004e-07 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035661+0000", 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.685e-06 2024-09-17T23:32:15.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035673+0000", 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671000000000001e-05 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035805+0000", 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013225800000000001 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038242+0000", 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002436676 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038630+0000", 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038801299999999999 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.038791+0000", 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016109300000000001 2024-09-17T23:32:15.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047922+0000", 2024-09-17T23:32:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091312310000000001 2024-09-17T23:32:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047954+0000", 2024-09-17T23:32:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1477999999999999e-05 2024-09-17T23:32:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048000+0000", 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5760000000000002e-05 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.048026+0000", 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6710999999999999e-05 2024-09-17T23:32:15.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:14829 3.11 3:8cedd9b8:::rbd_data.1152c1142907.000000000000004d:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:15.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 155.629901881, 2024-09-17T23:32:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014307263000000001, 2024-09-17T23:32:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14829 2024-09-17T23:32:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035829+0000", 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-07 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035833+0000", 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7790000000000002e-06 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035843+0000", 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8630000000000003e-06 2024-09-17T23:32:15.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035850+0000", 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3699999999999997e-06 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041730+0000", 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058792080000000004 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.041848+0000", 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118186 2024-09-17T23:32:15.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.042116+0000", 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026799300000000001 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.049813+0000", 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076969669999999999 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050040+0000", 2024-09-17T23:32:15.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022681200000000001 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050100+0000", 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0044999999999999e-05 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.050136+0000", 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6368999999999997e-05 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4737.0:14750 3.1 e41/33 3:81753cc8:::rbd_data.1152c1142907.0000000000000024:head v 41'192, mlcod=39'189)", 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 155.629862011, 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012049687, 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035869+0000", 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-09-17T23:32:15.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035874+0000", 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035882+0000", 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8900000000000007e-06 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.035888+0000", 2024-09-17T23:32:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037716+0000", 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018277079999999999 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.037735+0000", 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9258000000000001e-05 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047888+0000", 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010152714 2024-09-17T23:32:15.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.047918+0000", 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0817e-05 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4737.0:21698 3.11 3:8e158006:::rbd_data.1152c1142907.0000000000000153:head [delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 155.335357784, 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011357692000000001, 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4737", 2024-09-17T23:32:15.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21698 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330373+0000", 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330374+0000", 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5799999999999998e-07 2024-09-17T23:32:15.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330379+0000", 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9629999999999997e-06 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330392+0000", 2024-09-17T23:32:15.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3463e-05 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.330400+0000", 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9910000000000002e-06 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334713+0000", 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043130809999999999 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.334899+0000", 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018573799999999999 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:15.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.335122+0000", 2024-09-17T23:32:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022331799999999999 2024-09-17T23:32:15.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.339472+0000", 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043497090000000002 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341636+0000", 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021638640000000002 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341687+0000", 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1514999999999997e-05 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:40.341731+0000", 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3092e-05 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4694.0:921 3.1e e43/30 3:7fdf4885:::rbd_data.1152dc9ca65d.0000000000000002:head v 43'74, mlcod=43'72)", 2024-09-17T23:32:15.706 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 150.087656235, 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015198183000000001, 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578074+0000", 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.578075+0000", 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.23e-07 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587570+0000", 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094951849999999997 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587582+0000", 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1926000000000001e-05 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587590+0000", 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1510000000000004e-06 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587657+0000", 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6099999999999994e-05 2024-09-17T23:32:15.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.587678+0000", 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.158e-05 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593227+0000", 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055488619999999999 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:45.593273+0000", 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5655999999999998e-05 2024-09-17T23:32:15.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12635 3.14 e48/30 3:29eddeed:::rbd_data.115262cdab6.0000000000000007:head v 48'100, mlcod=48'98)", 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.077723969, 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026178554999999999, 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588007+0000", 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.588009+0000", 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1119999999999999e-06 2024-09-17T23:32:15.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598104+0000", 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010094888 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598127+0000", 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3371999999999999e-05 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598139+0000", 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2292e-05 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598218+0000", 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8207999999999994e-05 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.598241+0000", 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3249000000000001e-05 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614115+0000", 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015874652 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.614185+0000", 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9782000000000004e-05 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4968.0:12614 3.2 e48/30 3:4410fa08:::rbd_data.115262cdab6.0000000000000005:head v 48'132, mlcod=45'130)", 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 142.06449692000001, 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052981427999999997, 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601234+0000", 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.601235+0000", 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.24e-06 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641178+0000", 2024-09-17T23:32:15.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.039943287000000001 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641202+0000", 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836e-05 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641212+0000", 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0034000000000001e-05 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641275+0000", 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2531000000000002e-05 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.641297+0000", 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2478000000000001e-05 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654158+0000", 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012861262 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:29:53.654215+0000", 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6759999999999999e-05 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:290 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.52396776399999, 2024-09-17T23:32:15.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011029071, 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 290 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:15.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141763+0000", 2024-09-17T23:32:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-09-17T23:32:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141765+0000", 2024-09-17T23:32:15.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-09-17T23:32:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141772+0000", 2024-09-17T23:32:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0939999999999996e-06 2024-09-17T23:32:15.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141777+0000", 2024-09-17T23:32:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.322e-06 2024-09-17T23:32:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142150+0000", 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000373023 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142154+0000", 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152556+0000", 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010402042 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152694+0000", 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000137353 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152792+0000", 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412000000000006e-05 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:291 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.52394637399999, 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011190597, 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 291 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141784+0000", 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141785+0000", 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141787+0000", 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-06 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141792+0000", 2024-09-17T23:32:15.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1109999999999999e-06 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141796+0000", 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0280000000000002e-06 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142184+0000", 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038789400000000001 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142188+0000", 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2130000000000002e-06 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152821+0000", 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010632505 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152898+0000", 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7249999999999994e-05 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.152975+0000", 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7014999999999994e-05 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:292 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.52392924500001, 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011329339000000001, 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 292 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141801+0000", 2024-09-17T23:32:15.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141802+0000", 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141803+0000", 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7060000000000001e-06 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141809+0000", 2024-09-17T23:32:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141813+0000", 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2649999999999998e-06 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142214+0000", 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040136600000000001 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142218+0000", 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9740000000000004e-06 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153002+0000", 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010783802 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153062+0000", 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0389000000000001e-05 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153131+0000", 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8299000000000005e-05 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:293 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 2719744~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.52391196299999, 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011501684, 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 293 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141819+0000", 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1100000000000002e-07 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141821+0000", 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9460000000000001e-06 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141827+0000", 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141830+0000", 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9539999999999998e-06 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142906+0000", 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001075833 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142912+0000", 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153158+0000", 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010246197 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153214+0000", 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5628999999999999e-05 2024-09-17T23:32:15.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153320+0000", 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106363 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:294 3.11 3:89d0d418:::rbd_data.1152e108e789.0000000000000025:14 [read 3112960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.523894934, 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011675794, 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 294 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141836+0000", 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7300000000000002e-07 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141838+0000", 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1100000000000001e-06 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141843+0000", 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2079999999999999e-06 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141848+0000", 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2989999999999998e-06 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142959+0000", 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001110967 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142964+0000", 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6099999999999997e-06 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153351+0000", 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010386842 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153435+0000", 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4165999999999993e-05 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153512+0000", 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6318999999999999e-05 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:306 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3964928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.52386206, 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011786212000000001, 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 306 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141869+0000", 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141872+0000", 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6340000000000001e-06 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141879+0000", 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.141884+0000", 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143008+0000", 2024-09-17T23:32:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011239360000000001 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143013+0000", 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153545+0000", 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010532056 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153609+0000", 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4682000000000002e-05 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153655+0000", 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5521e-05 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5014.0:344 3.11 3:8c66fad3:::rbd_data.1152e108e789.0000000000000003:14 [read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 135.52327435399999, 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011416043000000001, 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5014", 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 344 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142456+0000", 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142457+0000", 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.23e-07 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142462+0000", 2024-09-17T23:32:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5689999999999998e-06 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142472+0000", 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0389e-05 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.142489+0000", 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6767999999999999e-05 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143038+0000", 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00054890699999999995 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.143042+0000", 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153690+0000", 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010647761 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153755+0000", 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5046e-05 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:00.153872+0000", 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011733 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:706 3.e e58/36 3:758b17d2:::rbd_object_map.1152ce7c866:head v 58'215, mlcod=58'211)", 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.501482213, 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013296654999999999, 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164248+0000", 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164249+0000", 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0500000000000003e-07 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164254+0000", 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.843e-06 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164262+0000", 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164267+0000", 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5000000000000001e-06 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164320+0000", 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3151e-05 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164331+0000", 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0983e-05 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013146292 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177545+0000", 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8065000000000006e-05 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:707 3.1 e58/33 3:81f44528:::rbd_data.1152ce7c866.000000000000012a:head v 58'308, mlcod=55'306)", 2024-09-17T23:32:15.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.50094502499999, 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012756857999999999, 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:15.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164786+0000", 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7100000000000001e-07 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164926+0000", 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014015100000000001 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164933+0000", 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2980000000000003e-06 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164937+0000", 2024-09-17T23:32:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-09-17T23:32:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164972+0000", 2024-09-17T23:32:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959000000000002e-05 2024-09-17T23:32:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.164985+0000", 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2506e-05 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177477+0000", 2024-09-17T23:32:15.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012492707 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177543+0000", 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5164000000000002e-05 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.5092.0:708 3.0 e58/30 3:0456156a:::rbd_data.1152ce7c866.0000000000000154:head v 58'198, mlcod=55'196)", 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 128.500458909, 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012569642000000001, 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165272+0000", 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.98e-07 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165397+0000", 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012533800000000001 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165404+0000", 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5220000000000004e-06 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165409+0000", 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165453+0000", 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4112e-05 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.165467+0000", 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4292e-05 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177795+0000", 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012327938 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:07.177841+0000", 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6545999999999998e-05 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3404 3.11 3:8caaecbf:::rbd_data.115217ef83d2.000000000000006d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.428316307, 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033377099, 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3404 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237414+0000", 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237415+0000", 2024-09-17T23:32:15.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2100000000000002e-07 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237417+0000", 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237423+0000", 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-09-17T23:32:15.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237428+0000", 2024-09-17T23:32:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.887e-06 2024-09-17T23:32:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239001+0000", 2024-09-17T23:32:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015732560000000001 2024-09-17T23:32:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239075+0000", 2024-09-17T23:32:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3659999999999996e-05 2024-09-17T23:32:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239254+0000", 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179242 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.244457+0000", 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052031580000000003 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270689+0000", 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026231925999999999 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270745+0000", 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6110000000000003e-05 2024-09-17T23:32:15.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270791+0000", 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6017999999999999e-05 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3411 3.11 3:8df0900d:::rbd_data.115217ef83d2.0000000000000071:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.42829449600001, 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033475355999999998, 2024-09-17T23:32:15.785 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3411 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237436+0000", 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237437+0000", 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237439+0000", 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-09-17T23:32:15.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237444+0000", 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.237449+0000", 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239762+0000", 2024-09-17T23:32:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023128139999999998 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.239876+0000", 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011425999999999999 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.270912+0000", 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031035301000000001 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.5250.0:3911 3.11 3:8ed67a32:::rbd_data.115217ef83d2.0000000000000075:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 119.41885162600001, 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032188992999999999, 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.5250", 2024-09-17T23:32:15.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.139:0/2529043501", 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 3911 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246879+0000", 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246881+0000", 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246886+0000", 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5319999999999997e-06 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246899+0000", 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2867999999999999e-05 2024-09-17T23:32:15.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246907+0000", 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0949999999999996e-06 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.246960+0000", 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3034000000000001e-05 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247037+0000", 2024-09-17T23:32:15.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6961999999999999e-05 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.247209+0000", 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000171817 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.249265+0000", 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020560600000000002 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:15.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.278921+0000", 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029655747 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279016+0000", 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4956000000000004e-05 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-17T23:30:16.279068+0000", 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2253000000000003e-05 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-17T23:32:15.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-17T23:32:15.796 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:15.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:15.798 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:15.795+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:15.862 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-17T23:32:15.898 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:15.896+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:15.927 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:15.927 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:15.928 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.928 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_repop(client.4434.0:28029 3.c e172/19 3:36bb075d:::rbd_header.1152fda325c3:head v 172'504, mlcod=172'502)", 2024-09-17T23:32:15.928 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:32:15.923799+0000", 2024-09-17T23:32:15.928 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0018770729999999999, 2024-09-17T23:32:15.928 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018961640000000001, 2024-09-17T23:32:15.928 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:15.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:15.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:15.929 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:15.929 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.929 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:15.929 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923799+0000", 2024-09-17T23:32:15.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:15.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.930 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.930 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:15.930 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923799+0000", 2024-09-17T23:32:15.930 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:15.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.931 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.931 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:15.931 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923800+0000", 2024-09-17T23:32:15.931 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9299999999999998e-07 2024-09-17T23:32:15.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923805+0000", 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4110000000000004e-06 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923821+0000", 2024-09-17T23:32:15.932 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.5872999999999999e-05 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923828+0000", 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7769999999999997e-06 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:15.933 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923876+0000", 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8458999999999998e-05 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.923900+0000", 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3121000000000001e-05 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.925664+0000", 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001764065 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:15.935 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:32:15.925696+0000", 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.1865000000000001e-05 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-17T23:32:15.936 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 1 2024-09-17T23:32:15.937 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:15.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:15.999 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:15.997+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:16.054 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:16.054 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:16.054 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:16.055 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:16.055 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:16.063 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:16.104 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:16.102+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:16.201 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:16.199+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.211 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:324 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 524288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.33364084199999, 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0204367910000001, 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 324 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.212 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874746+0000", 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874747+0000", 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9999999999999997e-07 2024-09-17T23:32:16.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874751+0000", 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7079999999999999e-06 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874761+0000", 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.91e-06 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874766+0000", 2024-09-17T23:32:16.214 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.412e-06 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874801+0000", 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4798000000000001e-05 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.874804+0000", 2024-09-17T23:32:16.215 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4039999999999999e-06 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882449+0000", 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.007644306 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.882459+0000", 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0338e-05 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.216 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.894889+0000", 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0124306059999999 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895049+0000", 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00015938200000000001 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895183+0000", 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000134227 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:340 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 0~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.318015712, 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0050985589999999, 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 340 2024-09-17T23:32:16.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890371+0000", 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890373+0000", 2024-09-17T23:32:16.219 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1510000000000001e-06 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890381+0000", 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3680000000000002e-06 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890391+0000", 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0437e-05 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.220 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890397+0000", 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8220000000000003e-06 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890458+0000", 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0919999999999999e-05 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7620000000000001e-06 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.221 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895327+0000", 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.004859862 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895380+0000", 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3545999999999999e-05 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895470+0000", 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9691000000000006e-05 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.222 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:341 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 512~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.31797805599999, 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005231312, 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 341 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.224 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890409+0000", 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4400000000000001e-07 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890411+0000", 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.978e-06 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890418+0000", 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2400000000000004e-06 2024-09-17T23:32:16.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890421+0000", 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.4970000000000001e-06 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890500+0000", 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8647999999999996e-05 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890506+0000", 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1199999999999999e-06 2024-09-17T23:32:16.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895502+0000", 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049965030000001 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895553+0000", 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0838999999999999e-05 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895640+0000", 2024-09-17T23:32:16.227 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7143000000000002e-05 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:342 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1024~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.31796062500001, 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005393859, 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.228 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 342 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890426+0000", 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890427+0000", 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3699999999999999e-07 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890428+0000", 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.234e-06 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890436+0000", 2024-09-17T23:32:16.230 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3389999999999993e-06 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890440+0000", 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5719999999999999e-06 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890536+0000", 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6224000000000006e-05 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.231 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890541+0000", 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4710000000000001e-06 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895676+0000", 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005135329 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895722+0000", 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6359000000000001e-05 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.232 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895820+0000", 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8094e-05 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:343 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.317925694, 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00554082, 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.233 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 343 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890461+0000", 2024-09-17T23:32:16.234 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890462+0000", 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.9499999999999998e-07 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890463+0000", 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.666e-06 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890467+0000", 2024-09-17T23:32:16.235 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0389999999999998e-06 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890470+0000", 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7379999999999999e-06 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890570+0000", 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9790000000000005e-05 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.236 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890575+0000", 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1819999999999997e-06 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895852+0000", 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005276882 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.895913+0000", 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1058999999999999e-05 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.237 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896002+0000", 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9169e-05 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:344 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.317894274, 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0056778209999999, 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.238 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 344 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:16.239 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890493+0000", 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.6e-07 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890498+0000", 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.459e-06 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.240 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890502+0000", 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9979999999999997e-06 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890505+0000", 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.6629999999999999e-06 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890604+0000", 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8615000000000004e-05 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.241 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890609+0000", 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1100000000000002e-06 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896033+0000", 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0054239380000001 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896084+0000", 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.1229e-05 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.242 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896171+0000", 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6549000000000001e-05 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:345 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.31787764800001, 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.00583567, 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.243 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 345 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890509+0000", 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890510+0000", 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890511+0000", 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.17e-06 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890518+0000", 2024-09-17T23:32:16.245 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.866e-06 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890521+0000", 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2040000000000002e-06 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890635+0000", 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000114123 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890640+0000", 2024-09-17T23:32:16.246 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0699999999999997e-06 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896201+0000", 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0055608629999999 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896258+0000", 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6672000000000002e-05 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896345+0000", 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7453000000000004e-05 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:346 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.31785595400001, 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.005982401, 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 346 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890531+0000", 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5600000000000002e-07 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890533+0000", 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.635e-06 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890537+0000", 2024-09-17T23:32:16.250 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5250000000000001e-06 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890539+0000", 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.852e-06 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890668+0000", 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000128231 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890672+0000", 2024-09-17T23:32:16.251 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0110000000000002e-06 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896376+0000", 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057042309999999 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896426+0000", 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0529999999999999e-05 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.252 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896514+0000", 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7130000000000001e-05 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4481.0:347 3.1e 3:7d3229e3:::rbd_data.11521d8bcd2d.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 169.31784423600001, 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0061509150000001, 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4481", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/1993410593", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 347 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:16.254 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890543+0000", 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0699999999999999e-07 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890544+0000", 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.204e-06 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.255 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890548+0000", 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8399999999999997e-06 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890551+0000", 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4509999999999999e-06 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890690+0000", 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013937500000000001 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.256 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:26.890695+0000", 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8980000000000004e-06 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896545+0000", 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0058506119999999 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896604+0000", 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8653999999999998e-05 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:27.896694+0000", 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9673999999999996e-05 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:502 3.1e 3:7e1ce722:::rbd_data.115240362710.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.35870571700001, 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0082824939999999, 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 502 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.259 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849681+0000", 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849683+0000", 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4610000000000001e-06 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849690+0000", 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0829999999999998e-06 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849705+0000", 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.523e-05 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849714+0000", 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4039999999999995e-06 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849784+0000", 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0529000000000002e-05 2024-09-17T23:32:16.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op must wait for map", 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.849792+0000", 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8080000000000005e-06 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853227+0000", 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.003434603 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853236+0000", 2024-09-17T23:32:16.262 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7149999999999993e-06 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.854947+0000", 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0017102769999999 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855070+0000", 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012382300000000001 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for subops from 1", 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855275+0000", 2024-09-17T23:32:16.263 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00020479100000000001 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "op_commit", 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857754+0000", 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0024784350000000002 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "sub_op_commit_rec", 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857862+0000", 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000108757 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "commit_sent", 2024-09-17T23:32:16.264 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857917+0000", 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4849000000000003e-05 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857964+0000", 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6728999999999999e-05 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:510 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:16.265 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.354750553, 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0032047449999999, 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 510 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:16.266 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853637+0000", 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853638+0000", 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0049999999999999e-06 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.267 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853642+0000", 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5530000000000003e-06 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853652+0000", 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.9469999999999998e-06 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853659+0000", 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.3050000000000004e-06 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.268 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853702+0000", 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2735000000000003e-05 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853708+0000", 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7130000000000002e-06 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856637+0000", 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029288329999999 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.269 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856745+0000", 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107895 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856841+0000", 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6759e-05 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.270 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:511 3.2 3:414ba27b:::rbd_data.1152adcf8b1e.0000000000000002:1b [sparse-read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.354709895, 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033719160000001, 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 511 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.271 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853677+0000", 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853678+0000", 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0500000000000003e-07 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.272 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853681+0000", 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.066e-06 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853690+0000", 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7509999999999997e-06 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853699+0000", 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.4730000000000006e-06 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.273 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853723+0000", 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4199999999999999e-05 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853727+0000", 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9570000000000004e-06 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856883+0000", 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0031558549999999 2024-09-17T23:32:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856942+0000", 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8872000000000001e-05 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857049+0000", 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000107137 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.275 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:524 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.354414248, 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0029059229999999, 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 524 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.276 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853973+0000", 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7199999999999999e-07 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.277 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853976+0000", 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.8770000000000002e-06 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853984+0000", 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9589999999999995e-06 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853990+0000", 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9070000000000004e-06 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.278 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854086+0000", 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.6291000000000004e-05 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854095+0000", 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856628+0000", 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002533039 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.279 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856756+0000", 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012816600000000001 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856879+0000", 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012235300000000001 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.280 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:525 3.1e 3:7b44572c:::rbd_data.1152adcf8b1e.0000000000000009:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.354381953, 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.003182792, 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 525 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.281 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854006+0000", 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0500000000000004e-07 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.282 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854008+0000", 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.807e-06 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854016+0000", 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8439999999999998e-06 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854022+0000", 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.5550000000000002e-06 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.283 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854121+0000", 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8632e-05 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854128+0000", 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4580000000000004e-06 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856921+0000", 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0027932450000001 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.284 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7245999999999998e-05 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857188+0000", 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0001795 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:537 3.11 3:8b011386:::rbd_data.1152adcf8b1e.000000000000001e:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.285 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.354227142, 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0014670720000001, 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 537 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.286 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4300000000000003e-07 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.287 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854164+0000", 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.191e-06 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854170+0000", 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.7059999999999998e-06 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854315+0000", 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014505799999999999 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.288 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854363+0000", 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7262000000000003e-05 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854369+0000", 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.6479999999999997e-06 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855422+0000", 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0010526310000001 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.289 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855538+0000", 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011585400000000001 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.855627+0000", 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9178999999999994e-05 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:12998 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140329332257936 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:16.290 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.353940822, 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.01213098, 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 12998 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:16.291 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854446+0000", 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854448+0000", 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.663e-06 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.292 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854456+0000", 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7810000000000005e-06 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854470+0000", 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3857e-05 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854478+0000", 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4549999999999995e-06 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.293 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854564+0000", 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5495999999999996e-05 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854577+0000", 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3127e-05 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864180+0000", 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0096031270000001 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.294 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864313+0000", 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013366699999999999 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866577+0000", 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0022638070000000001 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:99 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [call lock.get_info in=12b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:16.295 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.353872161, 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0105503339999999, 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 99 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:16.296 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854515+0000", 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854516+0000", 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1340000000000001e-06 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.297 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854522+0000", 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1680000000000001e-06 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854534+0000", 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1731e-05 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854542+0000", 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4749999999999993e-06 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.298 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854683+0000", 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014021300000000001 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854696+0000", 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3705e-05 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864846+0000", 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0101499190000001 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.299 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.864923+0000", 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.6762999999999997e-05 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865065+0000", 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014222599999999999 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.300 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5367.0:101 3.1 3:84a9d67e:::rbd_header.11523b91a3f4:head [watch reconnect cookie 140367517277712 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 110.353830187, 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0122273690000001, 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5367", 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3230583415", 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 101 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.301 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854557+0000", 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854558+0000", 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9100000000000003e-07 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.302 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854562+0000", 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2270000000000004e-06 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854573+0000", 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0611000000000001e-05 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854580+0000", 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2200000000000003e-06 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.303 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854803+0000", 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000223692 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:25.854822+0000", 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.8760999999999999e-05 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865092+0000", 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0102696870000001 2024-09-17T23:32:16.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.865155+0000", 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2595000000000006e-05 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:26.866784+0000", 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0016297849999999999 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.305 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:21993 3.19 3:9b2301af:::rbd_header.1152c10d4b9c:head [watch reconnect cookie 140329534356960 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 55.354046164000003, 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2846123999999999, 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 21993 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.306 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854341+0000", 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854343+0000", 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.7120000000000001e-06 2024-09-17T23:32:16.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854349+0000", 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.1410000000000002e-06 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854360+0000", 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1467e-05 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.308 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854370+0000", 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.2590000000000006e-06 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854434+0000", 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4113000000000001e-05 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:20.854444+0000", 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0118999999999999e-05 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.309 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134085+0000", 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2796412340000001 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.134167+0000", 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.2489000000000003e-05 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136330+0000", 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0021621069999999999 2024-09-17T23:32:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.136445+0000", 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000115445 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:22.138953+0000", 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0025083140000000002 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.311 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:24856 3.b 3:d3034e8c:::rbd_header.1152ea45e50a:head [watch reconnect cookie 140329332260848 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 23.202726827999999, 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0448234409999999, 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 24856 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.312 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.313 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.313 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:16.313 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005660+0000", 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005662+0000", 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9350000000000001e-06 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005669+0000", 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3470000000000004e-06 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005683+0000", 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.4419999999999999e-05 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.005694+0000", 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0917999999999999e-05 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006157+0000", 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00046328800000000001 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:53.006166+0000", 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.4489999999999999e-06 2024-09-17T23:32:16.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041545+0000", 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.035378916 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for rw locks", 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.041630+0000", 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5628000000000001e-05 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045500+0000", 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0038699910000000001 2024-09-17T23:32:16.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.045553+0000", 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2516000000000003e-05 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:54.050484+0000", 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0049310329999999996 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.320 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.321 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:16.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:16.322 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:16.320+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:16.423 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:16.421+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:16.470 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:16.470 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:16.470 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:16.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:16.524 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:16.522+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:16.586 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "complaint_time": 30, 2024-09-17T23:32:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: "num_blocked_ops": 0 2024-09-17T23:32:16.586 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:16.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:16.636 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:16.634+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:16.725 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:16.724+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:16.743 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:16.743 INFO:teuthology.orchestra.run.smithi037.stdout: "size": 20, 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 600, 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [ 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2109 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1124352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.246629945, 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049266891, 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.744 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2109 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494015+0000", 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.745 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494016+0000", 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.9099999999999999e-07 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494018+0000", 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.2400000000000002e-06 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494024+0000", 2024-09-17T23:32:16.746 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4099999999999996e-06 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494029+0000", 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.7940000000000002e-06 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543178+0000", 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049149117999999999 2024-09-17T23:32:16.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543223+0000", 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4594e-05 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543282+0000", 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9344e-05 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2110 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1133568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.246610065, 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049469503999999997, 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2110 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.749 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494035+0000", 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494036+0000", 2024-09-17T23:32:16.750 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.8299999999999998e-07 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494037+0000", 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.9709999999999998e-06 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494044+0000", 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2809999999999997e-06 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.751 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494048+0000", 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.5580000000000002e-06 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543318+0000", 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049269724000000001 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543406+0000", 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8191999999999999e-05 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.752 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543505+0000", 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8394999999999996e-05 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2111 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1142784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.246588527, 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049578283000000001, 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2111 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.754 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.494057+0000", 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495816+0000", 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0017595219999999999 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495820+0000", 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.7299999999999999e-06 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495828+0000", 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.1009999999999992e-06 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495833+0000", 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.853e-06 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543539+0000", 2024-09-17T23:32:16.756 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.047705742000000002 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543583+0000", 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4187999999999998e-05 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.543635+0000", 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2147000000000001e-05 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2120 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1225728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.24469581299999, 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049370672999999997, 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2120 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495949+0000", 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495950+0000", 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.0900000000000001e-07 2024-09-17T23:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495951+0000", 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0920000000000001e-06 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495955+0000", 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6e-06 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495958+0000", 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.146e-06 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545206+0000", 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049247468000000003 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545248+0000", 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2057000000000001e-05 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545320+0000", 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2101000000000006e-05 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2121 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1234944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.24468358300001, 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049488160000000003, 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2121 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495962+0000", 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.17e-07 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495963+0000", 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3239999999999999e-06 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.765 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495966+0000", 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2949999999999998e-06 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495969+0000", 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5210000000000001e-06 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545358+0000", 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049388768 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545400+0000", 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1795999999999998e-05 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545450+0000", 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0238999999999998e-05 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.767 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2122 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1244160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.244673014, 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049611934000000003, 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2122 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.768 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495972+0000", 2024-09-17T23:32:16.769 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.3099999999999999e-07 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495973+0000", 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0440000000000001e-06 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495977+0000", 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.207e-06 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.770 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495979+0000", 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4669999999999998e-06 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545475+0000", 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049495852 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545530+0000", 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4941999999999997e-05 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.771 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545584+0000", 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4190999999999997e-05 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2123 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1253376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.24466312600001, 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049810868000000001, 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2123 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.773 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495982+0000", 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.4900000000000002e-07 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495984+0000", 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1930000000000001e-06 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.774 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495987+0000", 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2150000000000001e-06 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495989+0000", 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5560000000000002e-06 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545628+0000", 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049638619000000002 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.775 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545716+0000", 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7732000000000006e-05 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545793+0000", 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.7304000000000004e-05 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2124 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1262592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.776 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.244652844, 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049932450000000003, 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2124 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.777 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495992+0000", 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495993+0000", 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5699999999999998e-07 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.778 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.495998+0000", 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7799999999999997e-06 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496002+0000", 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.5480000000000002e-06 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496005+0000", 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7360000000000001e-06 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545831+0000", 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.049825995999999997 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545872+0000", 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1170000000000001e-05 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545925+0000", 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.2862999999999999e-05 2024-09-17T23:32:16.780 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4737.0:2125 3.8 3:15a85a75:::rbd_data.115258a86553.0000000000000003:b [read 1271808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 159.24463695700001, 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.050052346999999997, 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.781 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4737", 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3885031727", 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 2125 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.496008+0000", 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497816+0000", 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.0018079120000000001 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497822+0000", 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.3569999999999997e-06 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497831+0000", 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1030000000000008e-06 2024-09-17T23:32:16.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.497838+0000", 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.374e-06 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.545968+0000", 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.048129863000000002 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.784 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546010+0000", 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.1891999999999998e-05 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:37.546061+0000", 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.0846000000000001e-05 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5001.0:4453 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 138.70571164, 2024-09-17T23:32:16.785 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.009079813, 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5001", 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3984108543", 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 4453 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034934+0000", 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7400000000000002e-07 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034939+0000", 2024-09-17T23:32:16.787 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6299999999999997e-06 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034948+0000", 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7339999999999997e-06 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034954+0000", 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2530000000000001e-06 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.788 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.034999+0000", 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4607999999999997e-05 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.035004+0000", 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7450000000000001e-06 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042310+0000", 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.007305363 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.789 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042457+0000", 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000146825 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043640+0000", 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001183637 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.043726+0000", 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.5459999999999998e-05 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.790 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044013+0000", 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00028768400000000002 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:7501 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 138.70364225399999, 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0072744760000001, 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 7501 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.792 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037003+0000", 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037004+0000", 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0500000000000002e-07 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037008+0000", 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3000000000000003e-06 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.793 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037017+0000", 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7490000000000004e-06 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037023+0000", 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4869999999999999e-06 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037068+0000", 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.4459999999999998e-05 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.794 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:58.037072+0000", 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3209999999999998e-06 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042868+0000", 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0057957879999999 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for missing object", 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.042915+0000", 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.6993999999999998e-05 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.795 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044069+0000", 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.001153935 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044140+0000", 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1178000000000002e-05 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.044277+0000", 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013735900000000001 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.796 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5014.0:156 3.4 3:2070849a:::rbd_data.11521666c10.000000000000003a:head [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 137.371774886, 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78170134999999996, 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5014", 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/240853589", 2024-09-17T23:32:16.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 156 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368870+0000", 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.798 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368871+0000", 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9800000000000002e-07 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368876+0000", 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.442e-06 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368888+0000", 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2320000000000001e-05 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.799 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368895+0000", 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4649999999999997e-06 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368960+0000", 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.4950000000000007e-05 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:29:59.368973+0000", 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.3069e-05 2024-09-17T23:32:16.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150440+0000", 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.78146702700000004 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:00.150572+0000", 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00013117900000000001 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:528 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.886810208, 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.002790584, 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 528 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853835+0000", 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.9500000000000003e-07 2024-09-17T23:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853838+0000", 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.96e-06 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853847+0000", 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7299999999999994e-06 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853853+0000", 2024-09-17T23:32:16.804 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4890000000000003e-06 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853910+0000", 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7630999999999998e-05 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853919+0000", 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3000000000000002e-06 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.805 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856374+0000", 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0024550029999999 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856485+0000", 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000111258 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856626+0000", 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00014071799999999999 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.806 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:529 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 3997696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.886777838, 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0033194160000001, 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 529 2024-09-17T23:32:16.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853867+0000", 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.808 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853868+0000", 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.2500000000000001e-07 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853870+0000", 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5979999999999999e-06 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.853877+0000", 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.9460000000000001e-06 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.809 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854005+0000", 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000127572 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854370+0000", 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00036474199999999999 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854377+0000", 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.2830000000000004e-06 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.810 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856990+0000", 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0026128350000001 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857081+0000", 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.0972999999999998e-05 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857187+0000", 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000106042 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.811 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:536 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.88648546100001, 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0023996230000001, 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.812 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 536 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854160+0000", 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854161+0000", 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8999999999999995e-07 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854165+0000", 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.3930000000000001e-06 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854190+0000", 2024-09-17T23:32:16.814 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.5415e-05 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854197+0000", 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.1509999999999998e-06 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854257+0000", 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.9012999999999999e-05 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.815 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.854265+0000", 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.9420000000000001e-06 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856337+0000", 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.0020713349999999 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856462+0000", 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00012474099999999999 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.816 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856559+0000", 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7843000000000005e-05 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:549 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.880661323, 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99743811299999996, 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 549 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.818 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859984+0000", 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.6700000000000003e-07 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859988+0000", 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2569999999999999e-06 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.819 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.859996+0000", 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3529999999999995e-06 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860002+0000", 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.2319999999999998e-06 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.189e-05 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.820 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860051+0000", 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.8730000000000001e-06 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857222+0000", 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99717069300000005 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857308+0000", 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6687000000000005e-05 2024-09-17T23:32:16.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857422+0000", 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.00011356100000000001 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:560 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 360448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.88061982000001, 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99695608400000002, 2024-09-17T23:32:16.822 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 560 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860025+0000", 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860026+0000", 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.7700000000000004e-07 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860029+0000", 2024-09-17T23:32:16.824 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.348e-06 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860038+0000", 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.3979999999999999e-06 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860044+0000", 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8420000000000001e-06 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860098+0000", 2024-09-17T23:32:16.825 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.4576000000000001e-05 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.860108+0000", 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.8390000000000002e-06 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856826+0000", 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.99671809499999997 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.826 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856884+0000", 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 5.8001999999999998e-05 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.856981+0000", 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.7406999999999998e-05 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:574 3.8 3:154b5b79:::rbd_data.1152adcf8b1e.000000000000000b:1b [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:16.827 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.837881553, 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95491373700000004, 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 574 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:16.828 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902764+0000", 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902765+0000", 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.59e-06 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.829 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902773+0000", 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.4529999999999996e-06 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902800+0000", 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 2.7659999999999999e-05 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902841+0000", 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.0695000000000003e-05 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.830 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902905+0000", 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.3891000000000002e-05 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902913+0000", 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.9470000000000003e-06 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857459+0000", 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95454619500000004 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.831 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857546+0000", 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.6466999999999997e-05 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857677+0000", 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.000131839 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.5325.0:585 3.4 3:2068a91b:::rbd_data.1152adcf8b1e.000000000000001d:1b [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 115.837779212, 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95430134700000002, 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "started", 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.5325", 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/3070755003", 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 585 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.833 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902866+0000", 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.834 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902867+0000", 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0100000000000005e-07 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902870+0000", 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 3.2499999999999998e-06 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902879+0000", 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.7590000000000003e-06 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.835 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902886+0000", 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.5939999999999996e-06 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902934+0000", 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000001e-05 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:20.902939+0000", 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8760000000000004e-06 2024-09-17T23:32:16.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857008+0000", 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.95406904400000003 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "started", 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857079+0000", 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.0963999999999993e-05 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:30:21.857167+0000", 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 8.8009000000000004e-05 2024-09-17T23:32:16.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "description": "osd_op(client.4434.0:23425 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e135)", 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "initiated_at": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 37.857465310000002, 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98117612200000004, 2024-09-17T23:32:16.838 INFO:teuthology.orchestra.run.smithi037.stdout: "continuous": false, 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type_data": { 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flag_point": "reached pg", 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client_info": { 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client": "client.4434", 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "client_addr": "172.21.15.139:0/435459966", 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tid": 23425 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "events": [ 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "initiated", 2024-09-17T23:32:16.839 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "header_read", 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883180+0000", 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "throttled", 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883181+0000", 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 9.1100000000000004e-07 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.840 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "all_read", 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883186+0000", 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 4.8250000000000004e-06 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "dispatched", 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883198+0000", 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.2424e-05 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "queued_for_pg", 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883204+0000", 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 6.0669999999999997e-06 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883283+0000", 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.8971e-05 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "waiting for peered", 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:38.883290+0000", 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 7.25e-06 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "reached_pg", 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864345+0000", 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 0.98105423000000003 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-17T23:32:16.842 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "event": "done", 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "time": "2024-09-17T23:31:39.864356+0000", 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: "duration": 1.1443999999999999e-05 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-17T23:32:16.843 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:16.845 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:16.845 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:16.843+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:16.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T23:32:16.945 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:16.943+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-17T23:32:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "ops": [], 2024-09-17T23:32:17.002 INFO:teuthology.orchestra.run.smithi037.stdout: "num_ops": 0 2024-09-17T23:32:17.002 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-17T23:32:17.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:17.046 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:17.044+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:17.052 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-17T23:32:17.052 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T23:32:17.069 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-17T23:32:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T23:32:17.085 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-17T23:32:17.086 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T23:32:17.104 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-17T23:32:17.104 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T23:32:17.121 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-17T23:32:17.121 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T23:32:17.138 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-17T23:32:17.138 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T23:32:17.147 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-17T23:32:17.146+0000 7fda188de640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32233) UID: 0 2024-09-17T23:32:17.160 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.160 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.169 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:17.248 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:17.246+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:17.277 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.277 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T23:32:17.349 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:17.347+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:17.394 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.394 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T23:32:17.450 INFO:tasks.ceph.osd.5.smithi037.stderr:2024-09-17T23:32:17.449+0000 7f6a2d9d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 44446) UID: 0 2024-09-17T23:32:17.509 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.509 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T23:32:17.551 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:17.549+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:17.625 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.625 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T23:32:17.653 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-09-17T23:32:17.651+0000 7f9b88260640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32000) UID: 0 2024-09-17T23:32:17.742 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.743 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T23:32:17.753 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-17T23:32:17.751+0000 7f26d430a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32237) UID: 0 2024-09-17T23:32:17.855 INFO:tasks.ceph.osd.4.smithi037.stderr:2024-09-17T23:32:17.853+0000 7f788f3e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32002) UID: 0 2024-09-17T23:32:17.859 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.859 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T23:32:17.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T23:32:17.957 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-17T23:32:17.956+0000 7fabe2e84640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32240) UID: 0 2024-09-17T23:32:17.981 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:17.981 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:17.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T23:32:17.992 INFO:teuthology.orchestra.run.smithi139.stdout:All operations completed A-OK! 2024-09-17T23:32:18.002 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:18.003 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T23:32:18.012 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T23:32:18.034 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T23:32:18.043 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-17T23:32:18.043 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-17T23:32:18.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-17T23:32:18.144 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.145 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T23:32:18.273 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.273 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T23:32:18.390 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.390 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T23:32:18.399 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:32:18.397+0000 7f21c94c5640 -1 mon.a@0(leader).osd e174 definitely_dead 0 2024-09-17T23:32:18.519 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.519 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T23:32:18.635 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.635 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T23:32:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.751 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T23:32:18.867 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.867 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.870 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-17T23:32:18.872 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.1. 2024-09-17T23:32:18.878 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T23:32:18.890 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-17T23:32:18.890 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-17T23:32:18.890 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-17T23:32:18.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T23:32:18.985 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:18.985 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:18.993 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T23:32:19.101 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:19.101 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:19.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T23:32:19.217 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:19.217 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:19.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T23:32:19.233 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-17T23:32:19.232+0000 7f21c94c5640 -1 mon.a@0(leader).osd e175 definitely_dead 0 2024-09-17T23:32:19.333 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:19.333 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:19.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T23:32:19.448 INFO:teuthology.orchestra.run.smithi037.stdout:ERROR: (22) Invalid argument 2024-09-17T23:32:19.448 INFO:teuthology.orchestra.run.smithi037.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T23:32:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.2. 2024-09-17T23:32:19.895 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-17T23:32:19.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T23:32:20.016 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-17T23:32:20.016 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T23:32:20.025 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-17T23:32:20.026 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T23:32:20.034 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-17T23:32:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T23:32:20.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T23:32:20.865 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T23:32:20.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T23:32:20.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T23:32:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-17T23:32:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T23:32:21.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T23:32:21.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T23:32:22.871 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-17T23:32:22.872 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T23:32:22.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T23:32:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T23:32:23.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T23:32:24.876 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T23:32:24.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T23:32:25.862 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T23:32:25.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T23:32:26.864 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T23:32:26.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T23:32:27.918 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T23:32:27.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T23:32:28.923 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T23:32:29.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T23:32:29.933 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T23:32:29.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T23:32:30.942 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T23:32:30.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T23:32:31.946 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T23:32:31.964 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T23:32:31.964 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T23:32:31.964 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T23:32:31.964 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T23:32:31.965 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T23:32:31.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:32:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:32.260 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":188,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:32:31.941241+0000","last_up_change":"2024-09-17T23:31:11.038388+0000","last_in_change":"2024-09-17T23:32:19.868974+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T23:28:58.406361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T23:29:03.876212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T23:29:14.532518+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":174,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6817","nonce":2793055955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6819","nonce":2793055955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6823","nonce":2793055955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6821","nonce":2793055955}]},"public_addr":"172.21.15.31:6817/2793055955","cluster_addr":"172.21.15.31:6819/2793055955","heartbeat_back_addr":"172.21.15.31:6823/2793055955","heartbeat_front_addr":"172.21.15.31:6821/2793055955","state":["exists","up"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6809","nonce":2784091455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6811","nonce":2784091455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6815","nonce":2784091455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6813","nonce":2784091455}]},"public_addr":"172.21.15.31:6809/2784091455","cluster_addr":"172.21.15.31:6811/2784091455","heartbeat_back_addr":"172.21.15.31:6815/2784091455","heartbeat_front_addr":"172.21.15.31:6813/2784091455","state":["exists","up"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6801","nonce":464841260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6803","nonce":464841260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6807","nonce":464841260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6805","nonce":464841260}]},"public_addr":"172.21.15.31:6801/464841260","cluster_addr":"172.21.15.31:6803/464841260","heartbeat_back_addr":"172.21.15.31:6807/464841260","heartbeat_front_addr":"172.21.15.31:6805/464841260","state":["exists","up"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6801","nonce":1436186932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6803","nonce":1436186932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6807","nonce":1436186932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6805","nonce":1436186932}]},"public_addr":"172.21.15.37:6801/1436186932","cluster_addr":"172.21.15.37:6803/1436186932","heartbeat_back_addr":"172.21.15.37:6807/1436186932","heartbeat_front_addr":"172.21.15.37:6805/1436186932","state":["exists","up"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6817","nonce":154081689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6819","nonce":154081689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6823","nonce":154081689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6821","nonce":154081689}]},"public_addr":"172.21.15.37:6817/154081689","cluster_addr":"172.21.15.37:6819/154081689","heartbeat_back_addr":"172.21.15.37:6823/154081689","heartbeat_front_addr":"172.21.15.37:6821/154081689","state":["exists","up"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":93,"up_from":107,"up_thru":179,"down_at":94,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6809","nonce":1359318569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6811","nonce":1359318569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6815","nonce":1359318569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6813","nonce":1359318569}]},"public_addr":"172.21.15.37:6809/1359318569","cluster_addr":"172.21.15.37:6811/1359318569","heartbeat_back_addr":"172.21.15.37:6815/1359318569","heartbeat_front_addr":"172.21.15.37:6813/1359318569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:55.062494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.641591+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.788407+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.360450+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.895512+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.797662+0000","dead_epoch":94}],"pg_upmap":[{"pgid":"3.11","osds":[0,4]},{"pgid":"3.12","osds":[2,1]},{"pgid":"3.17","osds":[5,4]}],"pg_upmap_items":[{"pgid":"3.12","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:32:32.273 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T23:32:32.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:32:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:32.565 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":188,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:32:31.941241+0000","last_up_change":"2024-09-17T23:31:11.038388+0000","last_in_change":"2024-09-17T23:32:19.868974+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T23:28:58.406361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T23:29:03.876212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T23:29:14.532518+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":174,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6817","nonce":2793055955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6819","nonce":2793055955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6823","nonce":2793055955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6821","nonce":2793055955}]},"public_addr":"172.21.15.31:6817/2793055955","cluster_addr":"172.21.15.31:6819/2793055955","heartbeat_back_addr":"172.21.15.31:6823/2793055955","heartbeat_front_addr":"172.21.15.31:6821/2793055955","state":["exists","up"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6809","nonce":2784091455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6811","nonce":2784091455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6815","nonce":2784091455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6813","nonce":2784091455}]},"public_addr":"172.21.15.31:6809/2784091455","cluster_addr":"172.21.15.31:6811/2784091455","heartbeat_back_addr":"172.21.15.31:6815/2784091455","heartbeat_front_addr":"172.21.15.31:6813/2784091455","state":["exists","up"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6801","nonce":464841260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6803","nonce":464841260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6807","nonce":464841260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6805","nonce":464841260}]},"public_addr":"172.21.15.31:6801/464841260","cluster_addr":"172.21.15.31:6803/464841260","heartbeat_back_addr":"172.21.15.31:6807/464841260","heartbeat_front_addr":"172.21.15.31:6805/464841260","state":["exists","up"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6801","nonce":1436186932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6803","nonce":1436186932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6807","nonce":1436186932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6805","nonce":1436186932}]},"public_addr":"172.21.15.37:6801/1436186932","cluster_addr":"172.21.15.37:6803/1436186932","heartbeat_back_addr":"172.21.15.37:6807/1436186932","heartbeat_front_addr":"172.21.15.37:6805/1436186932","state":["exists","up"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6817","nonce":154081689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6819","nonce":154081689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6823","nonce":154081689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6821","nonce":154081689}]},"public_addr":"172.21.15.37:6817/154081689","cluster_addr":"172.21.15.37:6819/154081689","heartbeat_back_addr":"172.21.15.37:6823/154081689","heartbeat_front_addr":"172.21.15.37:6821/154081689","state":["exists","up"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":93,"up_from":107,"up_thru":179,"down_at":94,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6809","nonce":1359318569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6811","nonce":1359318569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6815","nonce":1359318569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6813","nonce":1359318569}]},"public_addr":"172.21.15.37:6809/1359318569","cluster_addr":"172.21.15.37:6811/1359318569","heartbeat_back_addr":"172.21.15.37:6815/1359318569","heartbeat_front_addr":"172.21.15.37:6813/1359318569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:55.062494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.641591+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.788407+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.360450+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.895512+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.797662+0000","dead_epoch":94}],"pg_upmap":[{"pgid":"3.11","osds":[0,4]},{"pgid":"3.12","osds":[2,1]},{"pgid":"3.17","osds":[5,4]}],"pg_upmap_items":[{"pgid":"3.12","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:32:32.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T23:32:32.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T23:32:32.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T23:32:32.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T23:32:32.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T23:32:32.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T23:32:32.737 INFO:teuthology.orchestra.run.smithi031.stdout:47244640302 2024-09-17T23:32:32.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:32:32.758 INFO:teuthology.orchestra.run.smithi031.stdout:47244640302 2024-09-17T23:32:32.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:32:32.790 INFO:teuthology.orchestra.run.smithi031.stdout:47244640302 2024-09-17T23:32:32.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:32:32.794 INFO:teuthology.orchestra.run.smithi031.stdout:47244640302 2024-09-17T23:32:32.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:32:32.800 INFO:teuthology.orchestra.run.smithi031.stdout:459561500690 2024-09-17T23:32:32.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:32:32.803 INFO:teuthology.orchestra.run.smithi031.stdout:47244640303 2024-09-17T23:32:32.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:32:33.163 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:33.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.3 2024-09-17T23:32:33.189 INFO:teuthology.orchestra.run.smithi031.stdout:459561500689 2024-09-17T23:32:33.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 459561500690 got 459561500689 for osd.5 2024-09-17T23:32:33.235 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:33.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.0 2024-09-17T23:32:33.257 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:33.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.2 2024-09-17T23:32:33.281 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:33.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.4 2024-09-17T23:32:33.310 INFO:teuthology.orchestra.run.smithi031.stdout:47244640302 2024-09-17T23:32:33.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640303 got 47244640302 for osd.1 2024-09-17T23:32:34.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:32:34.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:32:34.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:32:34.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:32:34.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:32:34.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:32:34.636 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:34.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.2 2024-09-17T23:32:34.683 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:34.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.4 2024-09-17T23:32:34.698 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:34.706 INFO:teuthology.orchestra.run.smithi031.stdout:459561500689 2024-09-17T23:32:34.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.3 2024-09-17T23:32:34.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 459561500690 got 459561500689 for osd.5 2024-09-17T23:32:34.739 INFO:teuthology.orchestra.run.smithi031.stdout:47244640301 2024-09-17T23:32:34.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640301 for osd.0 2024-09-17T23:32:34.809 INFO:teuthology.orchestra.run.smithi031.stdout:47244640302 2024-09-17T23:32:34.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640303 got 47244640302 for osd.1 2024-09-17T23:32:35.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T23:32:35.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T23:32:35.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T23:32:35.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T23:32:35.752 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T23:32:35.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T23:32:36.011 INFO:teuthology.orchestra.run.smithi031.stdout:47244640303 2024-09-17T23:32:36.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640303 for osd.2 2024-09-17T23:32:36.028 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:36.117 INFO:teuthology.orchestra.run.smithi031.stdout:459561500690 2024-09-17T23:32:36.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 459561500690 got 459561500690 for osd.5 2024-09-17T23:32:36.130 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:36.147 INFO:teuthology.orchestra.run.smithi031.stdout:47244640303 2024-09-17T23:32:36.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640303 for osd.4 2024-09-17T23:32:36.165 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:36.169 INFO:teuthology.orchestra.run.smithi031.stdout:47244640303 2024-09-17T23:32:36.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640303 for osd.3 2024-09-17T23:32:36.182 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:47244640303 2024-09-17T23:32:36.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640303 for osd.0 2024-09-17T23:32:36.247 INFO:teuthology.orchestra.run.smithi031.stdout:47244640304 2024-09-17T23:32:36.247 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:36.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640303 got 47244640304 for osd.1 2024-09-17T23:32:36.259 DEBUG:teuthology.parallel:result is None 2024-09-17T23:32:36.259 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T23:32:36.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:36.591 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:36.592 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:36.604 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-17T23:32:36.399530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19240,"ondisk_log_size":19240,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":326808,"kb_used_data":13372,"kb_used_omap":27,"kb_used_meta":313252,"kb_avail":565904232,"statfs":{"total":579820584960,"available":579485933568,"internally_reserved":0,"allocated":13692928,"data_stored":6939227,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":320770952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001238"},"pg_stats":[{"pgid":"3.1f","version":"174'465","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963140+0000","last_change":"2024-09-17T23:32:32.963140+0000","last_active":"2024-09-17T23:32:32.963140+0000","last_peered":"2024-09-17T23:32:32.963140+0000","last_clean":"2024-09-17T23:32:32.963140+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:32.963140+0000","last_undegraded":"2024-09-17T23:32:32.963140+0000","last_fullsized":"2024-09-17T23:32:32.963140+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:04.457427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"174'578","reported_seq":9351,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954468+0000","last_change":"2024-09-17T23:32:22.877385+0000","last_active":"2024-09-17T23:32:32.954468+0000","last_peered":"2024-09-17T23:32:32.954468+0000","last_clean":"2024-09-17T23:32:32.954468+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:32.954468+0000","last_undegraded":"2024-09-17T23:32:32.954468+0000","last_fullsized":"2024-09-17T23:32:32.954468+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:09.418470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"174'373","reported_seq":5985,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959626+0000","last_change":"2024-09-17T23:32:26.875330+0000","last_active":"2024-09-17T23:32:32.959626+0000","last_peered":"2024-09-17T23:32:32.959626+0000","last_clean":"2024-09-17T23:32:32.959626+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:32.959626+0000","last_undegraded":"2024-09-17T23:32:32.959626+0000","last_fullsized":"2024-09-17T23:32:32.959626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:18.413939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"174'471","reported_seq":9402,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960398+0000","last_change":"2024-09-17T23:32:24.889849+0000","last_active":"2024-09-17T23:32:32.960398+0000","last_peered":"2024-09-17T23:32:32.960398+0000","last_clean":"2024-09-17T23:32:32.960398+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:32.960398+0000","last_undegraded":"2024-09-17T23:32:32.960398+0000","last_fullsized":"2024-09-17T23:32:32.960398+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:36.156958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"173'452","reported_seq":19741,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954200+0000","last_change":"2024-09-17T23:32:17.870826+0000","last_active":"2024-09-17T23:32:32.954200+0000","last_peered":"2024-09-17T23:32:32.954200+0000","last_clean":"2024-09-17T23:32:32.954200+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:32.954200+0000","last_undegraded":"2024-09-17T23:32:32.954200+0000","last_fullsized":"2024-09-17T23:32:32.954200+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:57.276383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"169'882","reported_seq":7762,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960630+0000","last_change":"2024-09-17T23:32:22.881407+0000","last_active":"2024-09-17T23:32:32.960630+0000","last_peered":"2024-09-17T23:32:32.960630+0000","last_clean":"2024-09-17T23:32:32.960630+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:32.960630+0000","last_undegraded":"2024-09-17T23:32:32.960630+0000","last_fullsized":"2024-09-17T23:32:32.960630+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:41.917085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"173'844","reported_seq":6806,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299237+0000","last_change":"2024-09-17T23:32:17.877448+0000","last_active":"2024-09-17T23:32:33.299237+0000","last_peered":"2024-09-17T23:32:33.299237+0000","last_clean":"2024-09-17T23:32:33.299237+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:33.299237+0000","last_undegraded":"2024-09-17T23:32:33.299237+0000","last_fullsized":"2024-09-17T23:32:33.299237+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:25.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"170'487","reported_seq":4566,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954121+0000","last_change":"2024-09-17T23:32:17.869351+0000","last_active":"2024-09-17T23:32:32.954121+0000","last_peered":"2024-09-17T23:32:32.954121+0000","last_clean":"2024-09-17T23:32:32.954121+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:32.954121+0000","last_undegraded":"2024-09-17T23:32:32.954121+0000","last_fullsized":"2024-09-17T23:32:32.954121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:48.297102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"173'675","reported_seq":8334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955935+0000","last_change":"2024-09-17T23:32:17.874867+0000","last_active":"2024-09-17T23:32:32.955935+0000","last_peered":"2024-09-17T23:32:32.955935+0000","last_clean":"2024-09-17T23:32:32.955935+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:32.955935+0000","last_undegraded":"2024-09-17T23:32:32.955935+0000","last_fullsized":"2024-09-17T23:32:32.955935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:31.383792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"173'394","reported_seq":4573,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959820+0000","last_change":"2024-09-17T23:32:26.872510+0000","last_active":"2024-09-17T23:32:32.959820+0000","last_peered":"2024-09-17T23:32:32.959820+0000","last_clean":"2024-09-17T23:32:32.959820+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:32.959820+0000","last_undegraded":"2024-09-17T23:32:32.959820+0000","last_fullsized":"2024-09-17T23:32:32.959820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:36.272282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"174'635","reported_seq":13063,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954948+0000","last_change":"2024-09-17T23:32:17.872364+0000","last_active":"2024-09-17T23:32:32.954948+0000","last_peered":"2024-09-17T23:32:32.954948+0000","last_clean":"2024-09-17T23:32:32.954948+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:32.954948+0000","last_undegraded":"2024-09-17T23:32:32.954948+0000","last_fullsized":"2024-09-17T23:32:32.954948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:33.326848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'630","reported_seq":8364,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960280+0000","last_change":"2024-09-17T23:32:26.873492+0000","last_active":"2024-09-17T23:32:32.960280+0000","last_peered":"2024-09-17T23:32:32.960280+0000","last_clean":"2024-09-17T23:32:32.960280+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:32.960280+0000","last_undegraded":"2024-09-17T23:32:32.960280+0000","last_fullsized":"2024-09-17T23:32:32.960280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:01.374405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960092+0000","last_change":"2024-09-17T23:32:24.890813+0000","last_active":"2024-09-17T23:32:32.960092+0000","last_peered":"2024-09-17T23:32:32.960092+0000","last_clean":"2024-09-17T23:32:32.960092+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:32.960092+0000","last_undegraded":"2024-09-17T23:32:32.960092+0000","last_fullsized":"2024-09-17T23:32:32.960092+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:15.170449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"174'664","reported_seq":5270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299239+0000","last_change":"2024-09-17T23:32:17.874681+0000","last_active":"2024-09-17T23:32:33.299239+0000","last_peered":"2024-09-17T23:32:33.299239+0000","last_clean":"2024-09-17T23:32:33.299239+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:33.299239+0000","last_undegraded":"2024-09-17T23:32:33.299239+0000","last_fullsized":"2024-09-17T23:32:33.299239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:10.119882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299322+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:33.299322+0000","last_peered":"2024-09-17T23:32:33.299322+0000","last_clean":"2024-09-17T23:32:33.299322+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:33.299322+0000","last_undegraded":"2024-09-17T23:32:33.299322+0000","last_fullsized":"2024-09-17T23:32:33.299322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'453","reported_seq":12753,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299389+0000","last_change":"2024-09-17T23:32:17.875953+0000","last_active":"2024-09-17T23:32:33.299389+0000","last_peered":"2024-09-17T23:32:33.299389+0000","last_clean":"2024-09-17T23:32:33.299389+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:33.299389+0000","last_undegraded":"2024-09-17T23:32:33.299389+0000","last_fullsized":"2024-09-17T23:32:33.299389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:08.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959531+0000","last_change":"2024-09-17T23:32:24.883350+0000","last_active":"2024-09-17T23:32:32.959531+0000","last_peered":"2024-09-17T23:32:32.959531+0000","last_clean":"2024-09-17T23:32:32.959531+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:32.959531+0000","last_undegraded":"2024-09-17T23:32:32.959531+0000","last_fullsized":"2024-09-17T23:32:32.959531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:05.948442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"174'416","reported_seq":7427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299244+0000","last_change":"2024-09-17T23:32:17.875596+0000","last_active":"2024-09-17T23:32:33.299244+0000","last_peered":"2024-09-17T23:32:33.299244+0000","last_clean":"2024-09-17T23:32:33.299244+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:33.299244+0000","last_undegraded":"2024-09-17T23:32:33.299244+0000","last_fullsized":"2024-09-17T23:32:33.299244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:19.550098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.958724+0000","last_change":"2024-09-17T23:32:32.958724+0000","last_active":"2024-09-17T23:32:32.958724+0000","last_peered":"2024-09-17T23:32:32.958724+0000","last_clean":"2024-09-17T23:32:32.958724+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:32.958724+0000","last_undegraded":"2024-09-17T23:32:32.958724+0000","last_fullsized":"2024-09-17T23:32:32.958724+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:25.311067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"171'401","reported_seq":10668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959525+0000","last_change":"2024-09-17T23:32:26.872700+0000","last_active":"2024-09-17T23:32:32.959525+0000","last_peered":"2024-09-17T23:32:32.959525+0000","last_clean":"2024-09-17T23:32:32.959525+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:32.959525+0000","last_undegraded":"2024-09-17T23:32:32.959525+0000","last_fullsized":"2024-09-17T23:32:32.959525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:22.943613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"174'690","reported_seq":7987,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954629+0000","last_change":"2024-09-17T23:32:24.884398+0000","last_active":"2024-09-17T23:32:32.954629+0000","last_peered":"2024-09-17T23:32:32.954629+0000","last_clean":"2024-09-17T23:32:32.954629+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:32.954629+0000","last_undegraded":"2024-09-17T23:32:32.954629+0000","last_fullsized":"2024-09-17T23:32:32.954629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:18.501946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955603+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:32.955603+0000","last_peered":"2024-09-17T23:32:32.955603+0000","last_clean":"2024-09-17T23:32:32.955603+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:32.955603+0000","last_undegraded":"2024-09-17T23:32:32.955603+0000","last_fullsized":"2024-09-17T23:32:32.955603+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"172'699","reported_seq":12525,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963637+0000","last_change":"2024-09-17T23:32:32.963637+0000","last_active":"2024-09-17T23:32:32.963637+0000","last_peered":"2024-09-17T23:32:32.963637+0000","last_clean":"2024-09-17T23:32:32.963637+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:32.963637+0000","last_undegraded":"2024-09-17T23:32:32.963637+0000","last_fullsized":"2024-09-17T23:32:32.963637+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:06.933631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954229+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:32.954229+0000","last_peered":"2024-09-17T23:32:32.954229+0000","last_clean":"2024-09-17T23:32:32.954229+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:32.954229+0000","last_undegraded":"2024-09-17T23:32:32.954229+0000","last_fullsized":"2024-09-17T23:32:32.954229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"174'949","reported_seq":9718,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299168+0000","last_change":"2024-09-17T23:32:23.880388+0000","last_active":"2024-09-17T23:32:33.299168+0000","last_peered":"2024-09-17T23:32:33.299168+0000","last_clean":"2024-09-17T23:32:33.299168+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:33.299168+0000","last_undegraded":"2024-09-17T23:32:33.299168+0000","last_fullsized":"2024-09-17T23:32:33.299168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:19.966648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959403+0000","last_change":"2024-09-17T23:32:20.871165+0000","last_active":"2024-09-17T23:32:32.959403+0000","last_peered":"2024-09-17T23:32:32.959403+0000","last_clean":"2024-09-17T23:32:32.959403+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:32.959403+0000","last_undegraded":"2024-09-17T23:32:32.959403+0000","last_fullsized":"2024-09-17T23:32:32.959403+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"173'717","reported_seq":11659,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954280+0000","last_change":"2024-09-17T23:32:24.882213+0000","last_active":"2024-09-17T23:32:32.954280+0000","last_peered":"2024-09-17T23:32:32.954280+0000","last_clean":"2024-09-17T23:32:32.954280+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:32.954280+0000","last_undegraded":"2024-09-17T23:32:32.954280+0000","last_fullsized":"2024-09-17T23:32:32.954280+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:18.707223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":28,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959438+0000","last_change":"2024-09-17T23:32:24.887027+0000","last_active":"2024-09-17T23:32:32.959438+0000","last_peered":"2024-09-17T23:32:32.959438+0000","last_clean":"2024-09-17T23:32:32.959438+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:32.959438+0000","last_undegraded":"2024-09-17T23:32:32.959438+0000","last_fullsized":"2024-09-17T23:32:32.959438+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:50.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":454,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959142+0000","last_change":"2024-09-17T23:32:22.879076+0000","last_active":"2024-09-17T23:32:32.959142+0000","last_peered":"2024-09-17T23:32:32.959142+0000","last_clean":"2024-09-17T23:32:32.959142+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:32.959142+0000","last_undegraded":"2024-09-17T23:32:32.959142+0000","last_fullsized":"2024-09-17T23:32:32.959142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:01.370196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"174'491","reported_seq":15228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299131+0000","last_change":"2024-09-17T23:32:23.878098+0000","last_active":"2024-09-17T23:32:33.299131+0000","last_peered":"2024-09-17T23:32:33.299131+0000","last_clean":"2024-09-17T23:32:33.299131+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:33.299131+0000","last_undegraded":"2024-09-17T23:32:33.299131+0000","last_fullsized":"2024-09-17T23:32:33.299131+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.816169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"168'741","reported_seq":11750,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955122+0000","last_change":"2024-09-17T23:32:17.869242+0000","last_active":"2024-09-17T23:32:32.955122+0000","last_peered":"2024-09-17T23:32:32.955122+0000","last_clean":"2024-09-17T23:32:32.955122+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:32.955122+0000","last_undegraded":"2024-09-17T23:32:32.955122+0000","last_fullsized":"2024-09-17T23:32:32.955122+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:20.421701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'630","reported_seq":10164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955543+0000","last_change":"2024-09-17T23:32:26.875971+0000","last_active":"2024-09-17T23:32:32.955543+0000","last_peered":"2024-09-17T23:32:32.955543+0000","last_clean":"2024-09-17T23:32:32.955543+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:32.955543+0000","last_undegraded":"2024-09-17T23:32:32.955543+0000","last_fullsized":"2024-09-17T23:32:32.955543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:15.468503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"174'953","reported_seq":11755,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954268+0000","last_change":"2024-09-17T23:32:17.873346+0000","last_active":"2024-09-17T23:32:32.954268+0000","last_peered":"2024-09-17T23:32:32.954268+0000","last_clean":"2024-09-17T23:32:32.954268+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:32.954268+0000","last_undegraded":"2024-09-17T23:32:32.954268+0000","last_fullsized":"2024-09-17T23:32:32.954268+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:06:59.962287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'510","reported_seq":5624,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955422+0000","last_change":"2024-09-17T23:32:17.873091+0000","last_active":"2024-09-17T23:32:32.955422+0000","last_peered":"2024-09-17T23:32:32.955422+0000","last_clean":"2024-09-17T23:32:32.955422+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:32.955422+0000","last_undegraded":"2024-09-17T23:32:32.955422+0000","last_fullsized":"2024-09-17T23:32:32.955422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:28:21.353619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'436","reported_seq":9861,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959619+0000","last_change":"2024-09-17T23:32:22.878491+0000","last_active":"2024-09-17T23:32:32.959619+0000","last_peered":"2024-09-17T23:32:32.959619+0000","last_clean":"2024-09-17T23:32:32.959619+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:32.959619+0000","last_undegraded":"2024-09-17T23:32:32.959619+0000","last_fullsized":"2024-09-17T23:32:32.959619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:52.466160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"170'749","reported_seq":10350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959730+0000","last_change":"2024-09-17T23:32:26.876897+0000","last_active":"2024-09-17T23:32:32.959730+0000","last_peered":"2024-09-17T23:32:32.959730+0000","last_clean":"2024-09-17T23:32:32.959730+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:32.959730+0000","last_undegraded":"2024-09-17T23:32:32.959730+0000","last_fullsized":"2024-09-17T23:32:32.959730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:42.259419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"173'386","reported_seq":5130,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955226+0000","last_change":"2024-09-17T23:32:26.876451+0000","last_active":"2024-09-17T23:32:32.955226+0000","last_peered":"2024-09-17T23:32:32.955226+0000","last_clean":"2024-09-17T23:32:32.955226+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:32.955226+0000","last_undegraded":"2024-09-17T23:32:32.955226+0000","last_fullsized":"2024-09-17T23:32:32.955226+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:28.427580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"171'917","reported_seq":7998,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959251+0000","last_change":"2024-09-17T23:32:23.878627+0000","last_active":"2024-09-17T23:32:32.959251+0000","last_peered":"2024-09-17T23:32:32.959251+0000","last_clean":"2024-09-17T23:32:32.959251+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:32.959251+0000","last_undegraded":"2024-09-17T23:32:32.959251+0000","last_fullsized":"2024-09-17T23:32:32.959251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:13.949860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"173'380","reported_seq":8679,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955044+0000","last_change":"2024-09-17T23:32:17.871775+0000","last_active":"2024-09-17T23:32:32.955044+0000","last_peered":"2024-09-17T23:32:32.955044+0000","last_clean":"2024-09-17T23:32:32.955044+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:32.955044+0000","last_undegraded":"2024-09-17T23:32:32.955044+0000","last_fullsized":"2024-09-17T23:32:32.955044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:41.522918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"172'447","reported_seq":5925,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959067+0000","last_change":"2024-09-17T23:32:26.874708+0000","last_active":"2024-09-17T23:32:32.959067+0000","last_peered":"2024-09-17T23:32:32.959067+0000","last_clean":"2024-09-17T23:32:32.959067+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:32.959067+0000","last_undegraded":"2024-09-17T23:32:32.959067+0000","last_fullsized":"2024-09-17T23:32:32.959067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:23.493952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"174'605","reported_seq":15334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959350+0000","last_change":"2024-09-17T23:32:24.890401+0000","last_active":"2024-09-17T23:32:32.959350+0000","last_peered":"2024-09-17T23:32:32.959350+0000","last_clean":"2024-09-17T23:32:32.959350+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:32.959350+0000","last_undegraded":"2024-09-17T23:32:32.959350+0000","last_fullsized":"2024-09-17T23:32:32.959350+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:18.258055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19120,"ondisk_log_size":19120,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500691,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43140,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94328700,"statfs":{"total":96636764160,"available":96592588800,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640303,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51452,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320388,"statfs":{"total":96636764160,"available":96584077312,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640303,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77816,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640303,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37560,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334280,"statfs":{"total":96636764160,"available":96598302720,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2306048,"data_stored":1549280,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640303,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69508,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302332,"statfs":{"total":96636764160,"available":96565587968,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:36.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:36.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:36.896 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:36.907 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-17T23:32:36.399530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19240,"ondisk_log_size":19240,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":326808,"kb_used_data":13372,"kb_used_omap":27,"kb_used_meta":313252,"kb_avail":565904232,"statfs":{"total":579820584960,"available":579485933568,"internally_reserved":0,"allocated":13692928,"data_stored":6939227,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":320770952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001238"},"pg_stats":[{"pgid":"3.1f","version":"174'465","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963140+0000","last_change":"2024-09-17T23:32:32.963140+0000","last_active":"2024-09-17T23:32:32.963140+0000","last_peered":"2024-09-17T23:32:32.963140+0000","last_clean":"2024-09-17T23:32:32.963140+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:32.963140+0000","last_undegraded":"2024-09-17T23:32:32.963140+0000","last_fullsized":"2024-09-17T23:32:32.963140+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:04.457427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"174'578","reported_seq":9351,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954468+0000","last_change":"2024-09-17T23:32:22.877385+0000","last_active":"2024-09-17T23:32:32.954468+0000","last_peered":"2024-09-17T23:32:32.954468+0000","last_clean":"2024-09-17T23:32:32.954468+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:32.954468+0000","last_undegraded":"2024-09-17T23:32:32.954468+0000","last_fullsized":"2024-09-17T23:32:32.954468+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:09.418470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"174'373","reported_seq":5985,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959626+0000","last_change":"2024-09-17T23:32:26.875330+0000","last_active":"2024-09-17T23:32:32.959626+0000","last_peered":"2024-09-17T23:32:32.959626+0000","last_clean":"2024-09-17T23:32:32.959626+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:32.959626+0000","last_undegraded":"2024-09-17T23:32:32.959626+0000","last_fullsized":"2024-09-17T23:32:32.959626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:18.413939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"174'471","reported_seq":9402,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960398+0000","last_change":"2024-09-17T23:32:24.889849+0000","last_active":"2024-09-17T23:32:32.960398+0000","last_peered":"2024-09-17T23:32:32.960398+0000","last_clean":"2024-09-17T23:32:32.960398+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:32.960398+0000","last_undegraded":"2024-09-17T23:32:32.960398+0000","last_fullsized":"2024-09-17T23:32:32.960398+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:36.156958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"173'452","reported_seq":19741,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954200+0000","last_change":"2024-09-17T23:32:17.870826+0000","last_active":"2024-09-17T23:32:32.954200+0000","last_peered":"2024-09-17T23:32:32.954200+0000","last_clean":"2024-09-17T23:32:32.954200+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:32.954200+0000","last_undegraded":"2024-09-17T23:32:32.954200+0000","last_fullsized":"2024-09-17T23:32:32.954200+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:57.276383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"169'882","reported_seq":7762,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960630+0000","last_change":"2024-09-17T23:32:22.881407+0000","last_active":"2024-09-17T23:32:32.960630+0000","last_peered":"2024-09-17T23:32:32.960630+0000","last_clean":"2024-09-17T23:32:32.960630+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:32.960630+0000","last_undegraded":"2024-09-17T23:32:32.960630+0000","last_fullsized":"2024-09-17T23:32:32.960630+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:41.917085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"173'844","reported_seq":6806,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299237+0000","last_change":"2024-09-17T23:32:17.877448+0000","last_active":"2024-09-17T23:32:33.299237+0000","last_peered":"2024-09-17T23:32:33.299237+0000","last_clean":"2024-09-17T23:32:33.299237+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:33.299237+0000","last_undegraded":"2024-09-17T23:32:33.299237+0000","last_fullsized":"2024-09-17T23:32:33.299237+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:25.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"170'487","reported_seq":4566,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954121+0000","last_change":"2024-09-17T23:32:17.869351+0000","last_active":"2024-09-17T23:32:32.954121+0000","last_peered":"2024-09-17T23:32:32.954121+0000","last_clean":"2024-09-17T23:32:32.954121+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:32.954121+0000","last_undegraded":"2024-09-17T23:32:32.954121+0000","last_fullsized":"2024-09-17T23:32:32.954121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:48.297102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"173'675","reported_seq":8334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955935+0000","last_change":"2024-09-17T23:32:17.874867+0000","last_active":"2024-09-17T23:32:32.955935+0000","last_peered":"2024-09-17T23:32:32.955935+0000","last_clean":"2024-09-17T23:32:32.955935+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:32.955935+0000","last_undegraded":"2024-09-17T23:32:32.955935+0000","last_fullsized":"2024-09-17T23:32:32.955935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:31.383792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"173'394","reported_seq":4573,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959820+0000","last_change":"2024-09-17T23:32:26.872510+0000","last_active":"2024-09-17T23:32:32.959820+0000","last_peered":"2024-09-17T23:32:32.959820+0000","last_clean":"2024-09-17T23:32:32.959820+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:32.959820+0000","last_undegraded":"2024-09-17T23:32:32.959820+0000","last_fullsized":"2024-09-17T23:32:32.959820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:36.272282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"174'635","reported_seq":13063,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954948+0000","last_change":"2024-09-17T23:32:17.872364+0000","last_active":"2024-09-17T23:32:32.954948+0000","last_peered":"2024-09-17T23:32:32.954948+0000","last_clean":"2024-09-17T23:32:32.954948+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:32.954948+0000","last_undegraded":"2024-09-17T23:32:32.954948+0000","last_fullsized":"2024-09-17T23:32:32.954948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:33.326848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'630","reported_seq":8364,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960280+0000","last_change":"2024-09-17T23:32:26.873492+0000","last_active":"2024-09-17T23:32:32.960280+0000","last_peered":"2024-09-17T23:32:32.960280+0000","last_clean":"2024-09-17T23:32:32.960280+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:32.960280+0000","last_undegraded":"2024-09-17T23:32:32.960280+0000","last_fullsized":"2024-09-17T23:32:32.960280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:01.374405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960092+0000","last_change":"2024-09-17T23:32:24.890813+0000","last_active":"2024-09-17T23:32:32.960092+0000","last_peered":"2024-09-17T23:32:32.960092+0000","last_clean":"2024-09-17T23:32:32.960092+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:32.960092+0000","last_undegraded":"2024-09-17T23:32:32.960092+0000","last_fullsized":"2024-09-17T23:32:32.960092+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:15.170449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"174'664","reported_seq":5270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299239+0000","last_change":"2024-09-17T23:32:17.874681+0000","last_active":"2024-09-17T23:32:33.299239+0000","last_peered":"2024-09-17T23:32:33.299239+0000","last_clean":"2024-09-17T23:32:33.299239+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:33.299239+0000","last_undegraded":"2024-09-17T23:32:33.299239+0000","last_fullsized":"2024-09-17T23:32:33.299239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:10.119882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299322+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:33.299322+0000","last_peered":"2024-09-17T23:32:33.299322+0000","last_clean":"2024-09-17T23:32:33.299322+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:33.299322+0000","last_undegraded":"2024-09-17T23:32:33.299322+0000","last_fullsized":"2024-09-17T23:32:33.299322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'453","reported_seq":12753,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299389+0000","last_change":"2024-09-17T23:32:17.875953+0000","last_active":"2024-09-17T23:32:33.299389+0000","last_peered":"2024-09-17T23:32:33.299389+0000","last_clean":"2024-09-17T23:32:33.299389+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:33.299389+0000","last_undegraded":"2024-09-17T23:32:33.299389+0000","last_fullsized":"2024-09-17T23:32:33.299389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:08.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959531+0000","last_change":"2024-09-17T23:32:24.883350+0000","last_active":"2024-09-17T23:32:32.959531+0000","last_peered":"2024-09-17T23:32:32.959531+0000","last_clean":"2024-09-17T23:32:32.959531+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:32.959531+0000","last_undegraded":"2024-09-17T23:32:32.959531+0000","last_fullsized":"2024-09-17T23:32:32.959531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:05.948442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"174'416","reported_seq":7427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299244+0000","last_change":"2024-09-17T23:32:17.875596+0000","last_active":"2024-09-17T23:32:33.299244+0000","last_peered":"2024-09-17T23:32:33.299244+0000","last_clean":"2024-09-17T23:32:33.299244+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:33.299244+0000","last_undegraded":"2024-09-17T23:32:33.299244+0000","last_fullsized":"2024-09-17T23:32:33.299244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:19.550098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.958724+0000","last_change":"2024-09-17T23:32:32.958724+0000","last_active":"2024-09-17T23:32:32.958724+0000","last_peered":"2024-09-17T23:32:32.958724+0000","last_clean":"2024-09-17T23:32:32.958724+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:32.958724+0000","last_undegraded":"2024-09-17T23:32:32.958724+0000","last_fullsized":"2024-09-17T23:32:32.958724+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:25.311067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"171'401","reported_seq":10668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959525+0000","last_change":"2024-09-17T23:32:26.872700+0000","last_active":"2024-09-17T23:32:32.959525+0000","last_peered":"2024-09-17T23:32:32.959525+0000","last_clean":"2024-09-17T23:32:32.959525+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:32.959525+0000","last_undegraded":"2024-09-17T23:32:32.959525+0000","last_fullsized":"2024-09-17T23:32:32.959525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:22.943613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"174'690","reported_seq":7987,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954629+0000","last_change":"2024-09-17T23:32:24.884398+0000","last_active":"2024-09-17T23:32:32.954629+0000","last_peered":"2024-09-17T23:32:32.954629+0000","last_clean":"2024-09-17T23:32:32.954629+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:32.954629+0000","last_undegraded":"2024-09-17T23:32:32.954629+0000","last_fullsized":"2024-09-17T23:32:32.954629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:18.501946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955603+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:32.955603+0000","last_peered":"2024-09-17T23:32:32.955603+0000","last_clean":"2024-09-17T23:32:32.955603+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:32.955603+0000","last_undegraded":"2024-09-17T23:32:32.955603+0000","last_fullsized":"2024-09-17T23:32:32.955603+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"172'699","reported_seq":12525,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963637+0000","last_change":"2024-09-17T23:32:32.963637+0000","last_active":"2024-09-17T23:32:32.963637+0000","last_peered":"2024-09-17T23:32:32.963637+0000","last_clean":"2024-09-17T23:32:32.963637+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:32.963637+0000","last_undegraded":"2024-09-17T23:32:32.963637+0000","last_fullsized":"2024-09-17T23:32:32.963637+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:06.933631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954229+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:32.954229+0000","last_peered":"2024-09-17T23:32:32.954229+0000","last_clean":"2024-09-17T23:32:32.954229+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:32.954229+0000","last_undegraded":"2024-09-17T23:32:32.954229+0000","last_fullsized":"2024-09-17T23:32:32.954229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"174'949","reported_seq":9718,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299168+0000","last_change":"2024-09-17T23:32:23.880388+0000","last_active":"2024-09-17T23:32:33.299168+0000","last_peered":"2024-09-17T23:32:33.299168+0000","last_clean":"2024-09-17T23:32:33.299168+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:33.299168+0000","last_undegraded":"2024-09-17T23:32:33.299168+0000","last_fullsized":"2024-09-17T23:32:33.299168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:19.966648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959403+0000","last_change":"2024-09-17T23:32:20.871165+0000","last_active":"2024-09-17T23:32:32.959403+0000","last_peered":"2024-09-17T23:32:32.959403+0000","last_clean":"2024-09-17T23:32:32.959403+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:32.959403+0000","last_undegraded":"2024-09-17T23:32:32.959403+0000","last_fullsized":"2024-09-17T23:32:32.959403+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"173'717","reported_seq":11659,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954280+0000","last_change":"2024-09-17T23:32:24.882213+0000","last_active":"2024-09-17T23:32:32.954280+0000","last_peered":"2024-09-17T23:32:32.954280+0000","last_clean":"2024-09-17T23:32:32.954280+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:32.954280+0000","last_undegraded":"2024-09-17T23:32:32.954280+0000","last_fullsized":"2024-09-17T23:32:32.954280+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:18.707223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":28,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959438+0000","last_change":"2024-09-17T23:32:24.887027+0000","last_active":"2024-09-17T23:32:32.959438+0000","last_peered":"2024-09-17T23:32:32.959438+0000","last_clean":"2024-09-17T23:32:32.959438+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:32.959438+0000","last_undegraded":"2024-09-17T23:32:32.959438+0000","last_fullsized":"2024-09-17T23:32:32.959438+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:50.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":454,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959142+0000","last_change":"2024-09-17T23:32:22.879076+0000","last_active":"2024-09-17T23:32:32.959142+0000","last_peered":"2024-09-17T23:32:32.959142+0000","last_clean":"2024-09-17T23:32:32.959142+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:32.959142+0000","last_undegraded":"2024-09-17T23:32:32.959142+0000","last_fullsized":"2024-09-17T23:32:32.959142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:01.370196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"174'491","reported_seq":15228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299131+0000","last_change":"2024-09-17T23:32:23.878098+0000","last_active":"2024-09-17T23:32:33.299131+0000","last_peered":"2024-09-17T23:32:33.299131+0000","last_clean":"2024-09-17T23:32:33.299131+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:33.299131+0000","last_undegraded":"2024-09-17T23:32:33.299131+0000","last_fullsized":"2024-09-17T23:32:33.299131+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.816169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"168'741","reported_seq":11750,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955122+0000","last_change":"2024-09-17T23:32:17.869242+0000","last_active":"2024-09-17T23:32:32.955122+0000","last_peered":"2024-09-17T23:32:32.955122+0000","last_clean":"2024-09-17T23:32:32.955122+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:32.955122+0000","last_undegraded":"2024-09-17T23:32:32.955122+0000","last_fullsized":"2024-09-17T23:32:32.955122+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:20.421701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'630","reported_seq":10164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955543+0000","last_change":"2024-09-17T23:32:26.875971+0000","last_active":"2024-09-17T23:32:32.955543+0000","last_peered":"2024-09-17T23:32:32.955543+0000","last_clean":"2024-09-17T23:32:32.955543+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:32.955543+0000","last_undegraded":"2024-09-17T23:32:32.955543+0000","last_fullsized":"2024-09-17T23:32:32.955543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:15.468503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"174'953","reported_seq":11755,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954268+0000","last_change":"2024-09-17T23:32:17.873346+0000","last_active":"2024-09-17T23:32:32.954268+0000","last_peered":"2024-09-17T23:32:32.954268+0000","last_clean":"2024-09-17T23:32:32.954268+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:32.954268+0000","last_undegraded":"2024-09-17T23:32:32.954268+0000","last_fullsized":"2024-09-17T23:32:32.954268+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:06:59.962287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'510","reported_seq":5624,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955422+0000","last_change":"2024-09-17T23:32:17.873091+0000","last_active":"2024-09-17T23:32:32.955422+0000","last_peered":"2024-09-17T23:32:32.955422+0000","last_clean":"2024-09-17T23:32:32.955422+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:32.955422+0000","last_undegraded":"2024-09-17T23:32:32.955422+0000","last_fullsized":"2024-09-17T23:32:32.955422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:28:21.353619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'436","reported_seq":9861,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959619+0000","last_change":"2024-09-17T23:32:22.878491+0000","last_active":"2024-09-17T23:32:32.959619+0000","last_peered":"2024-09-17T23:32:32.959619+0000","last_clean":"2024-09-17T23:32:32.959619+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:32.959619+0000","last_undegraded":"2024-09-17T23:32:32.959619+0000","last_fullsized":"2024-09-17T23:32:32.959619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:52.466160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"170'749","reported_seq":10350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959730+0000","last_change":"2024-09-17T23:32:26.876897+0000","last_active":"2024-09-17T23:32:32.959730+0000","last_peered":"2024-09-17T23:32:32.959730+0000","last_clean":"2024-09-17T23:32:32.959730+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:32.959730+0000","last_undegraded":"2024-09-17T23:32:32.959730+0000","last_fullsized":"2024-09-17T23:32:32.959730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:42.259419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"173'386","reported_seq":5130,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955226+0000","last_change":"2024-09-17T23:32:26.876451+0000","last_active":"2024-09-17T23:32:32.955226+0000","last_peered":"2024-09-17T23:32:32.955226+0000","last_clean":"2024-09-17T23:32:32.955226+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:32.955226+0000","last_undegraded":"2024-09-17T23:32:32.955226+0000","last_fullsized":"2024-09-17T23:32:32.955226+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:28.427580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"171'917","reported_seq":7998,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959251+0000","last_change":"2024-09-17T23:32:23.878627+0000","last_active":"2024-09-17T23:32:32.959251+0000","last_peered":"2024-09-17T23:32:32.959251+0000","last_clean":"2024-09-17T23:32:32.959251+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:32.959251+0000","last_undegraded":"2024-09-17T23:32:32.959251+0000","last_fullsized":"2024-09-17T23:32:32.959251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:13.949860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"173'380","reported_seq":8679,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955044+0000","last_change":"2024-09-17T23:32:17.871775+0000","last_active":"2024-09-17T23:32:32.955044+0000","last_peered":"2024-09-17T23:32:32.955044+0000","last_clean":"2024-09-17T23:32:32.955044+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:32.955044+0000","last_undegraded":"2024-09-17T23:32:32.955044+0000","last_fullsized":"2024-09-17T23:32:32.955044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:41.522918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"172'447","reported_seq":5925,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959067+0000","last_change":"2024-09-17T23:32:26.874708+0000","last_active":"2024-09-17T23:32:32.959067+0000","last_peered":"2024-09-17T23:32:32.959067+0000","last_clean":"2024-09-17T23:32:32.959067+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:32.959067+0000","last_undegraded":"2024-09-17T23:32:32.959067+0000","last_fullsized":"2024-09-17T23:32:32.959067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:23.493952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"174'605","reported_seq":15334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959350+0000","last_change":"2024-09-17T23:32:24.890401+0000","last_active":"2024-09-17T23:32:32.959350+0000","last_peered":"2024-09-17T23:32:32.959350+0000","last_clean":"2024-09-17T23:32:32.959350+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:32.959350+0000","last_undegraded":"2024-09-17T23:32:32.959350+0000","last_fullsized":"2024-09-17T23:32:32.959350+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:18.258055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19120,"ondisk_log_size":19120,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500691,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43140,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94328700,"statfs":{"total":96636764160,"available":96592588800,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640303,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51452,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320388,"statfs":{"total":96636764160,"available":96584077312,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640303,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77816,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640303,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37560,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334280,"statfs":{"total":96636764160,"available":96598302720,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2306048,"data_stored":1549280,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640303,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69508,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302332,"statfs":{"total":96636764160,"available":96565587968,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:36.910 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T23:32:36.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T23:32:37.216 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T23:32:37.226 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T23:32:37.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:37.517 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:37.528 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-17T23:32:36.399530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19240,"ondisk_log_size":19240,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":326808,"kb_used_data":13372,"kb_used_omap":27,"kb_used_meta":313252,"kb_avail":565904232,"statfs":{"total":579820584960,"available":579485933568,"internally_reserved":0,"allocated":13692928,"data_stored":6939227,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":320770952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001238"},"pg_stats":[{"pgid":"3.1f","version":"174'465","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963140+0000","last_change":"2024-09-17T23:32:32.963140+0000","last_active":"2024-09-17T23:32:32.963140+0000","last_peered":"2024-09-17T23:32:32.963140+0000","last_clean":"2024-09-17T23:32:32.963140+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:32.963140+0000","last_undegraded":"2024-09-17T23:32:32.963140+0000","last_fullsized":"2024-09-17T23:32:32.963140+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:04.457427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"174'578","reported_seq":9351,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954468+0000","last_change":"2024-09-17T23:32:22.877385+0000","last_active":"2024-09-17T23:32:32.954468+0000","last_peered":"2024-09-17T23:32:32.954468+0000","last_clean":"2024-09-17T23:32:32.954468+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:32.954468+0000","last_undegraded":"2024-09-17T23:32:32.954468+0000","last_fullsized":"2024-09-17T23:32:32.954468+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:09.418470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"174'373","reported_seq":5985,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959626+0000","last_change":"2024-09-17T23:32:26.875330+0000","last_active":"2024-09-17T23:32:32.959626+0000","last_peered":"2024-09-17T23:32:32.959626+0000","last_clean":"2024-09-17T23:32:32.959626+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:32.959626+0000","last_undegraded":"2024-09-17T23:32:32.959626+0000","last_fullsized":"2024-09-17T23:32:32.959626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:18.413939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"174'471","reported_seq":9402,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960398+0000","last_change":"2024-09-17T23:32:24.889849+0000","last_active":"2024-09-17T23:32:32.960398+0000","last_peered":"2024-09-17T23:32:32.960398+0000","last_clean":"2024-09-17T23:32:32.960398+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:32.960398+0000","last_undegraded":"2024-09-17T23:32:32.960398+0000","last_fullsized":"2024-09-17T23:32:32.960398+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:36.156958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"173'452","reported_seq":19741,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954200+0000","last_change":"2024-09-17T23:32:17.870826+0000","last_active":"2024-09-17T23:32:32.954200+0000","last_peered":"2024-09-17T23:32:32.954200+0000","last_clean":"2024-09-17T23:32:32.954200+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:32.954200+0000","last_undegraded":"2024-09-17T23:32:32.954200+0000","last_fullsized":"2024-09-17T23:32:32.954200+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:57.276383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"169'882","reported_seq":7762,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960630+0000","last_change":"2024-09-17T23:32:22.881407+0000","last_active":"2024-09-17T23:32:32.960630+0000","last_peered":"2024-09-17T23:32:32.960630+0000","last_clean":"2024-09-17T23:32:32.960630+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:32.960630+0000","last_undegraded":"2024-09-17T23:32:32.960630+0000","last_fullsized":"2024-09-17T23:32:32.960630+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:41.917085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"173'844","reported_seq":6806,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299237+0000","last_change":"2024-09-17T23:32:17.877448+0000","last_active":"2024-09-17T23:32:33.299237+0000","last_peered":"2024-09-17T23:32:33.299237+0000","last_clean":"2024-09-17T23:32:33.299237+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:33.299237+0000","last_undegraded":"2024-09-17T23:32:33.299237+0000","last_fullsized":"2024-09-17T23:32:33.299237+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:25.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"170'487","reported_seq":4566,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954121+0000","last_change":"2024-09-17T23:32:17.869351+0000","last_active":"2024-09-17T23:32:32.954121+0000","last_peered":"2024-09-17T23:32:32.954121+0000","last_clean":"2024-09-17T23:32:32.954121+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:32.954121+0000","last_undegraded":"2024-09-17T23:32:32.954121+0000","last_fullsized":"2024-09-17T23:32:32.954121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:48.297102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"173'675","reported_seq":8334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955935+0000","last_change":"2024-09-17T23:32:17.874867+0000","last_active":"2024-09-17T23:32:32.955935+0000","last_peered":"2024-09-17T23:32:32.955935+0000","last_clean":"2024-09-17T23:32:32.955935+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:32.955935+0000","last_undegraded":"2024-09-17T23:32:32.955935+0000","last_fullsized":"2024-09-17T23:32:32.955935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:31.383792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"173'394","reported_seq":4573,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959820+0000","last_change":"2024-09-17T23:32:26.872510+0000","last_active":"2024-09-17T23:32:32.959820+0000","last_peered":"2024-09-17T23:32:32.959820+0000","last_clean":"2024-09-17T23:32:32.959820+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:32.959820+0000","last_undegraded":"2024-09-17T23:32:32.959820+0000","last_fullsized":"2024-09-17T23:32:32.959820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:36.272282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"174'635","reported_seq":13063,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954948+0000","last_change":"2024-09-17T23:32:17.872364+0000","last_active":"2024-09-17T23:32:32.954948+0000","last_peered":"2024-09-17T23:32:32.954948+0000","last_clean":"2024-09-17T23:32:32.954948+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:32.954948+0000","last_undegraded":"2024-09-17T23:32:32.954948+0000","last_fullsized":"2024-09-17T23:32:32.954948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:33.326848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'630","reported_seq":8364,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960280+0000","last_change":"2024-09-17T23:32:26.873492+0000","last_active":"2024-09-17T23:32:32.960280+0000","last_peered":"2024-09-17T23:32:32.960280+0000","last_clean":"2024-09-17T23:32:32.960280+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:32.960280+0000","last_undegraded":"2024-09-17T23:32:32.960280+0000","last_fullsized":"2024-09-17T23:32:32.960280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:01.374405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960092+0000","last_change":"2024-09-17T23:32:24.890813+0000","last_active":"2024-09-17T23:32:32.960092+0000","last_peered":"2024-09-17T23:32:32.960092+0000","last_clean":"2024-09-17T23:32:32.960092+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:32.960092+0000","last_undegraded":"2024-09-17T23:32:32.960092+0000","last_fullsized":"2024-09-17T23:32:32.960092+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:15.170449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"174'664","reported_seq":5270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299239+0000","last_change":"2024-09-17T23:32:17.874681+0000","last_active":"2024-09-17T23:32:33.299239+0000","last_peered":"2024-09-17T23:32:33.299239+0000","last_clean":"2024-09-17T23:32:33.299239+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:33.299239+0000","last_undegraded":"2024-09-17T23:32:33.299239+0000","last_fullsized":"2024-09-17T23:32:33.299239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:10.119882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299322+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:33.299322+0000","last_peered":"2024-09-17T23:32:33.299322+0000","last_clean":"2024-09-17T23:32:33.299322+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:33.299322+0000","last_undegraded":"2024-09-17T23:32:33.299322+0000","last_fullsized":"2024-09-17T23:32:33.299322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'453","reported_seq":12753,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299389+0000","last_change":"2024-09-17T23:32:17.875953+0000","last_active":"2024-09-17T23:32:33.299389+0000","last_peered":"2024-09-17T23:32:33.299389+0000","last_clean":"2024-09-17T23:32:33.299389+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:33.299389+0000","last_undegraded":"2024-09-17T23:32:33.299389+0000","last_fullsized":"2024-09-17T23:32:33.299389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:08.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959531+0000","last_change":"2024-09-17T23:32:24.883350+0000","last_active":"2024-09-17T23:32:32.959531+0000","last_peered":"2024-09-17T23:32:32.959531+0000","last_clean":"2024-09-17T23:32:32.959531+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:32.959531+0000","last_undegraded":"2024-09-17T23:32:32.959531+0000","last_fullsized":"2024-09-17T23:32:32.959531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:05.948442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"174'416","reported_seq":7427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299244+0000","last_change":"2024-09-17T23:32:17.875596+0000","last_active":"2024-09-17T23:32:33.299244+0000","last_peered":"2024-09-17T23:32:33.299244+0000","last_clean":"2024-09-17T23:32:33.299244+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:33.299244+0000","last_undegraded":"2024-09-17T23:32:33.299244+0000","last_fullsized":"2024-09-17T23:32:33.299244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:19.550098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.958724+0000","last_change":"2024-09-17T23:32:32.958724+0000","last_active":"2024-09-17T23:32:32.958724+0000","last_peered":"2024-09-17T23:32:32.958724+0000","last_clean":"2024-09-17T23:32:32.958724+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:32.958724+0000","last_undegraded":"2024-09-17T23:32:32.958724+0000","last_fullsized":"2024-09-17T23:32:32.958724+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:25.311067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"171'401","reported_seq":10668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959525+0000","last_change":"2024-09-17T23:32:26.872700+0000","last_active":"2024-09-17T23:32:32.959525+0000","last_peered":"2024-09-17T23:32:32.959525+0000","last_clean":"2024-09-17T23:32:32.959525+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:32.959525+0000","last_undegraded":"2024-09-17T23:32:32.959525+0000","last_fullsized":"2024-09-17T23:32:32.959525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:22.943613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"174'690","reported_seq":7987,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954629+0000","last_change":"2024-09-17T23:32:24.884398+0000","last_active":"2024-09-17T23:32:32.954629+0000","last_peered":"2024-09-17T23:32:32.954629+0000","last_clean":"2024-09-17T23:32:32.954629+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:32.954629+0000","last_undegraded":"2024-09-17T23:32:32.954629+0000","last_fullsized":"2024-09-17T23:32:32.954629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:18.501946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955603+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:32.955603+0000","last_peered":"2024-09-17T23:32:32.955603+0000","last_clean":"2024-09-17T23:32:32.955603+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:32.955603+0000","last_undegraded":"2024-09-17T23:32:32.955603+0000","last_fullsized":"2024-09-17T23:32:32.955603+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"172'699","reported_seq":12525,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963637+0000","last_change":"2024-09-17T23:32:32.963637+0000","last_active":"2024-09-17T23:32:32.963637+0000","last_peered":"2024-09-17T23:32:32.963637+0000","last_clean":"2024-09-17T23:32:32.963637+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:32.963637+0000","last_undegraded":"2024-09-17T23:32:32.963637+0000","last_fullsized":"2024-09-17T23:32:32.963637+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:06.933631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954229+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:32.954229+0000","last_peered":"2024-09-17T23:32:32.954229+0000","last_clean":"2024-09-17T23:32:32.954229+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:32.954229+0000","last_undegraded":"2024-09-17T23:32:32.954229+0000","last_fullsized":"2024-09-17T23:32:32.954229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"174'949","reported_seq":9718,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299168+0000","last_change":"2024-09-17T23:32:23.880388+0000","last_active":"2024-09-17T23:32:33.299168+0000","last_peered":"2024-09-17T23:32:33.299168+0000","last_clean":"2024-09-17T23:32:33.299168+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:33.299168+0000","last_undegraded":"2024-09-17T23:32:33.299168+0000","last_fullsized":"2024-09-17T23:32:33.299168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:19.966648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959403+0000","last_change":"2024-09-17T23:32:20.871165+0000","last_active":"2024-09-17T23:32:32.959403+0000","last_peered":"2024-09-17T23:32:32.959403+0000","last_clean":"2024-09-17T23:32:32.959403+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:32.959403+0000","last_undegraded":"2024-09-17T23:32:32.959403+0000","last_fullsized":"2024-09-17T23:32:32.959403+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"173'717","reported_seq":11659,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954280+0000","last_change":"2024-09-17T23:32:24.882213+0000","last_active":"2024-09-17T23:32:32.954280+0000","last_peered":"2024-09-17T23:32:32.954280+0000","last_clean":"2024-09-17T23:32:32.954280+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:32.954280+0000","last_undegraded":"2024-09-17T23:32:32.954280+0000","last_fullsized":"2024-09-17T23:32:32.954280+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:18.707223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":28,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959438+0000","last_change":"2024-09-17T23:32:24.887027+0000","last_active":"2024-09-17T23:32:32.959438+0000","last_peered":"2024-09-17T23:32:32.959438+0000","last_clean":"2024-09-17T23:32:32.959438+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:32.959438+0000","last_undegraded":"2024-09-17T23:32:32.959438+0000","last_fullsized":"2024-09-17T23:32:32.959438+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:50.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":454,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959142+0000","last_change":"2024-09-17T23:32:22.879076+0000","last_active":"2024-09-17T23:32:32.959142+0000","last_peered":"2024-09-17T23:32:32.959142+0000","last_clean":"2024-09-17T23:32:32.959142+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:32.959142+0000","last_undegraded":"2024-09-17T23:32:32.959142+0000","last_fullsized":"2024-09-17T23:32:32.959142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:01.370196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"174'491","reported_seq":15228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299131+0000","last_change":"2024-09-17T23:32:23.878098+0000","last_active":"2024-09-17T23:32:33.299131+0000","last_peered":"2024-09-17T23:32:33.299131+0000","last_clean":"2024-09-17T23:32:33.299131+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:33.299131+0000","last_undegraded":"2024-09-17T23:32:33.299131+0000","last_fullsized":"2024-09-17T23:32:33.299131+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.816169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"168'741","reported_seq":11750,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955122+0000","last_change":"2024-09-17T23:32:17.869242+0000","last_active":"2024-09-17T23:32:32.955122+0000","last_peered":"2024-09-17T23:32:32.955122+0000","last_clean":"2024-09-17T23:32:32.955122+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:32.955122+0000","last_undegraded":"2024-09-17T23:32:32.955122+0000","last_fullsized":"2024-09-17T23:32:32.955122+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:20.421701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'630","reported_seq":10164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955543+0000","last_change":"2024-09-17T23:32:26.875971+0000","last_active":"2024-09-17T23:32:32.955543+0000","last_peered":"2024-09-17T23:32:32.955543+0000","last_clean":"2024-09-17T23:32:32.955543+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:32.955543+0000","last_undegraded":"2024-09-17T23:32:32.955543+0000","last_fullsized":"2024-09-17T23:32:32.955543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:15.468503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"174'953","reported_seq":11755,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954268+0000","last_change":"2024-09-17T23:32:17.873346+0000","last_active":"2024-09-17T23:32:32.954268+0000","last_peered":"2024-09-17T23:32:32.954268+0000","last_clean":"2024-09-17T23:32:32.954268+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:32.954268+0000","last_undegraded":"2024-09-17T23:32:32.954268+0000","last_fullsized":"2024-09-17T23:32:32.954268+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:06:59.962287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'510","reported_seq":5624,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955422+0000","last_change":"2024-09-17T23:32:17.873091+0000","last_active":"2024-09-17T23:32:32.955422+0000","last_peered":"2024-09-17T23:32:32.955422+0000","last_clean":"2024-09-17T23:32:32.955422+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:32.955422+0000","last_undegraded":"2024-09-17T23:32:32.955422+0000","last_fullsized":"2024-09-17T23:32:32.955422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:28:21.353619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'436","reported_seq":9861,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959619+0000","last_change":"2024-09-17T23:32:22.878491+0000","last_active":"2024-09-17T23:32:32.959619+0000","last_peered":"2024-09-17T23:32:32.959619+0000","last_clean":"2024-09-17T23:32:32.959619+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:32.959619+0000","last_undegraded":"2024-09-17T23:32:32.959619+0000","last_fullsized":"2024-09-17T23:32:32.959619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:52.466160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"170'749","reported_seq":10350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959730+0000","last_change":"2024-09-17T23:32:26.876897+0000","last_active":"2024-09-17T23:32:32.959730+0000","last_peered":"2024-09-17T23:32:32.959730+0000","last_clean":"2024-09-17T23:32:32.959730+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:32.959730+0000","last_undegraded":"2024-09-17T23:32:32.959730+0000","last_fullsized":"2024-09-17T23:32:32.959730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:42.259419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"173'386","reported_seq":5130,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955226+0000","last_change":"2024-09-17T23:32:26.876451+0000","last_active":"2024-09-17T23:32:32.955226+0000","last_peered":"2024-09-17T23:32:32.955226+0000","last_clean":"2024-09-17T23:32:32.955226+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:32.955226+0000","last_undegraded":"2024-09-17T23:32:32.955226+0000","last_fullsized":"2024-09-17T23:32:32.955226+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:28.427580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"171'917","reported_seq":7998,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959251+0000","last_change":"2024-09-17T23:32:23.878627+0000","last_active":"2024-09-17T23:32:32.959251+0000","last_peered":"2024-09-17T23:32:32.959251+0000","last_clean":"2024-09-17T23:32:32.959251+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:32.959251+0000","last_undegraded":"2024-09-17T23:32:32.959251+0000","last_fullsized":"2024-09-17T23:32:32.959251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:13.949860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"173'380","reported_seq":8679,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955044+0000","last_change":"2024-09-17T23:32:17.871775+0000","last_active":"2024-09-17T23:32:32.955044+0000","last_peered":"2024-09-17T23:32:32.955044+0000","last_clean":"2024-09-17T23:32:32.955044+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:32.955044+0000","last_undegraded":"2024-09-17T23:32:32.955044+0000","last_fullsized":"2024-09-17T23:32:32.955044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:41.522918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"172'447","reported_seq":5925,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959067+0000","last_change":"2024-09-17T23:32:26.874708+0000","last_active":"2024-09-17T23:32:32.959067+0000","last_peered":"2024-09-17T23:32:32.959067+0000","last_clean":"2024-09-17T23:32:32.959067+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:32.959067+0000","last_undegraded":"2024-09-17T23:32:32.959067+0000","last_fullsized":"2024-09-17T23:32:32.959067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:23.493952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"174'605","reported_seq":15334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959350+0000","last_change":"2024-09-17T23:32:24.890401+0000","last_active":"2024-09-17T23:32:32.959350+0000","last_peered":"2024-09-17T23:32:32.959350+0000","last_clean":"2024-09-17T23:32:32.959350+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:32.959350+0000","last_undegraded":"2024-09-17T23:32:32.959350+0000","last_fullsized":"2024-09-17T23:32:32.959350+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:18.258055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19120,"ondisk_log_size":19120,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500691,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43140,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94328700,"statfs":{"total":96636764160,"available":96592588800,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640303,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51452,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320388,"statfs":{"total":96636764160,"available":96584077312,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640303,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77816,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640303,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37560,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334280,"statfs":{"total":96636764160,"available":96598302720,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2306048,"data_stored":1549280,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640303,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69508,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302332,"statfs":{"total":96636764160,"available":96565587968,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:37.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:37.818 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:37.832 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-17T23:32:36.399530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19240,"ondisk_log_size":19240,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":326808,"kb_used_data":13372,"kb_used_omap":27,"kb_used_meta":313252,"kb_avail":565904232,"statfs":{"total":579820584960,"available":579485933568,"internally_reserved":0,"allocated":13692928,"data_stored":6939227,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":320770952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001238"},"pg_stats":[{"pgid":"3.1f","version":"174'465","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963140+0000","last_change":"2024-09-17T23:32:32.963140+0000","last_active":"2024-09-17T23:32:32.963140+0000","last_peered":"2024-09-17T23:32:32.963140+0000","last_clean":"2024-09-17T23:32:32.963140+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:32.963140+0000","last_undegraded":"2024-09-17T23:32:32.963140+0000","last_fullsized":"2024-09-17T23:32:32.963140+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:04.457427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"174'578","reported_seq":9351,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954468+0000","last_change":"2024-09-17T23:32:22.877385+0000","last_active":"2024-09-17T23:32:32.954468+0000","last_peered":"2024-09-17T23:32:32.954468+0000","last_clean":"2024-09-17T23:32:32.954468+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:32.954468+0000","last_undegraded":"2024-09-17T23:32:32.954468+0000","last_fullsized":"2024-09-17T23:32:32.954468+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:09.418470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"174'373","reported_seq":5985,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959626+0000","last_change":"2024-09-17T23:32:26.875330+0000","last_active":"2024-09-17T23:32:32.959626+0000","last_peered":"2024-09-17T23:32:32.959626+0000","last_clean":"2024-09-17T23:32:32.959626+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:32.959626+0000","last_undegraded":"2024-09-17T23:32:32.959626+0000","last_fullsized":"2024-09-17T23:32:32.959626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:18.413939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"174'471","reported_seq":9402,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960398+0000","last_change":"2024-09-17T23:32:24.889849+0000","last_active":"2024-09-17T23:32:32.960398+0000","last_peered":"2024-09-17T23:32:32.960398+0000","last_clean":"2024-09-17T23:32:32.960398+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:32.960398+0000","last_undegraded":"2024-09-17T23:32:32.960398+0000","last_fullsized":"2024-09-17T23:32:32.960398+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:36.156958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"173'452","reported_seq":19741,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954200+0000","last_change":"2024-09-17T23:32:17.870826+0000","last_active":"2024-09-17T23:32:32.954200+0000","last_peered":"2024-09-17T23:32:32.954200+0000","last_clean":"2024-09-17T23:32:32.954200+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:32.954200+0000","last_undegraded":"2024-09-17T23:32:32.954200+0000","last_fullsized":"2024-09-17T23:32:32.954200+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:57.276383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"169'882","reported_seq":7762,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960630+0000","last_change":"2024-09-17T23:32:22.881407+0000","last_active":"2024-09-17T23:32:32.960630+0000","last_peered":"2024-09-17T23:32:32.960630+0000","last_clean":"2024-09-17T23:32:32.960630+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:32.960630+0000","last_undegraded":"2024-09-17T23:32:32.960630+0000","last_fullsized":"2024-09-17T23:32:32.960630+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:41.917085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"173'844","reported_seq":6806,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299237+0000","last_change":"2024-09-17T23:32:17.877448+0000","last_active":"2024-09-17T23:32:33.299237+0000","last_peered":"2024-09-17T23:32:33.299237+0000","last_clean":"2024-09-17T23:32:33.299237+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:33.299237+0000","last_undegraded":"2024-09-17T23:32:33.299237+0000","last_fullsized":"2024-09-17T23:32:33.299237+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:25.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"170'487","reported_seq":4566,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954121+0000","last_change":"2024-09-17T23:32:17.869351+0000","last_active":"2024-09-17T23:32:32.954121+0000","last_peered":"2024-09-17T23:32:32.954121+0000","last_clean":"2024-09-17T23:32:32.954121+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:32.954121+0000","last_undegraded":"2024-09-17T23:32:32.954121+0000","last_fullsized":"2024-09-17T23:32:32.954121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:48.297102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"173'675","reported_seq":8334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955935+0000","last_change":"2024-09-17T23:32:17.874867+0000","last_active":"2024-09-17T23:32:32.955935+0000","last_peered":"2024-09-17T23:32:32.955935+0000","last_clean":"2024-09-17T23:32:32.955935+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:32.955935+0000","last_undegraded":"2024-09-17T23:32:32.955935+0000","last_fullsized":"2024-09-17T23:32:32.955935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:31.383792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"173'394","reported_seq":4573,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959820+0000","last_change":"2024-09-17T23:32:26.872510+0000","last_active":"2024-09-17T23:32:32.959820+0000","last_peered":"2024-09-17T23:32:32.959820+0000","last_clean":"2024-09-17T23:32:32.959820+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:32.959820+0000","last_undegraded":"2024-09-17T23:32:32.959820+0000","last_fullsized":"2024-09-17T23:32:32.959820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:36.272282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"174'635","reported_seq":13063,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954948+0000","last_change":"2024-09-17T23:32:17.872364+0000","last_active":"2024-09-17T23:32:32.954948+0000","last_peered":"2024-09-17T23:32:32.954948+0000","last_clean":"2024-09-17T23:32:32.954948+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:32.954948+0000","last_undegraded":"2024-09-17T23:32:32.954948+0000","last_fullsized":"2024-09-17T23:32:32.954948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:33.326848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'630","reported_seq":8364,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960280+0000","last_change":"2024-09-17T23:32:26.873492+0000","last_active":"2024-09-17T23:32:32.960280+0000","last_peered":"2024-09-17T23:32:32.960280+0000","last_clean":"2024-09-17T23:32:32.960280+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:32.960280+0000","last_undegraded":"2024-09-17T23:32:32.960280+0000","last_fullsized":"2024-09-17T23:32:32.960280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:01.374405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960092+0000","last_change":"2024-09-17T23:32:24.890813+0000","last_active":"2024-09-17T23:32:32.960092+0000","last_peered":"2024-09-17T23:32:32.960092+0000","last_clean":"2024-09-17T23:32:32.960092+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:32.960092+0000","last_undegraded":"2024-09-17T23:32:32.960092+0000","last_fullsized":"2024-09-17T23:32:32.960092+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:15.170449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"174'664","reported_seq":5270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299239+0000","last_change":"2024-09-17T23:32:17.874681+0000","last_active":"2024-09-17T23:32:33.299239+0000","last_peered":"2024-09-17T23:32:33.299239+0000","last_clean":"2024-09-17T23:32:33.299239+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:33.299239+0000","last_undegraded":"2024-09-17T23:32:33.299239+0000","last_fullsized":"2024-09-17T23:32:33.299239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:10.119882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299322+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:33.299322+0000","last_peered":"2024-09-17T23:32:33.299322+0000","last_clean":"2024-09-17T23:32:33.299322+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:33.299322+0000","last_undegraded":"2024-09-17T23:32:33.299322+0000","last_fullsized":"2024-09-17T23:32:33.299322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'453","reported_seq":12753,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299389+0000","last_change":"2024-09-17T23:32:17.875953+0000","last_active":"2024-09-17T23:32:33.299389+0000","last_peered":"2024-09-17T23:32:33.299389+0000","last_clean":"2024-09-17T23:32:33.299389+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:33.299389+0000","last_undegraded":"2024-09-17T23:32:33.299389+0000","last_fullsized":"2024-09-17T23:32:33.299389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:08.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959531+0000","last_change":"2024-09-17T23:32:24.883350+0000","last_active":"2024-09-17T23:32:32.959531+0000","last_peered":"2024-09-17T23:32:32.959531+0000","last_clean":"2024-09-17T23:32:32.959531+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:32.959531+0000","last_undegraded":"2024-09-17T23:32:32.959531+0000","last_fullsized":"2024-09-17T23:32:32.959531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:05.948442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"174'416","reported_seq":7427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299244+0000","last_change":"2024-09-17T23:32:17.875596+0000","last_active":"2024-09-17T23:32:33.299244+0000","last_peered":"2024-09-17T23:32:33.299244+0000","last_clean":"2024-09-17T23:32:33.299244+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:33.299244+0000","last_undegraded":"2024-09-17T23:32:33.299244+0000","last_fullsized":"2024-09-17T23:32:33.299244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:19.550098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.958724+0000","last_change":"2024-09-17T23:32:32.958724+0000","last_active":"2024-09-17T23:32:32.958724+0000","last_peered":"2024-09-17T23:32:32.958724+0000","last_clean":"2024-09-17T23:32:32.958724+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:32.958724+0000","last_undegraded":"2024-09-17T23:32:32.958724+0000","last_fullsized":"2024-09-17T23:32:32.958724+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:25.311067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"171'401","reported_seq":10668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959525+0000","last_change":"2024-09-17T23:32:26.872700+0000","last_active":"2024-09-17T23:32:32.959525+0000","last_peered":"2024-09-17T23:32:32.959525+0000","last_clean":"2024-09-17T23:32:32.959525+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:32.959525+0000","last_undegraded":"2024-09-17T23:32:32.959525+0000","last_fullsized":"2024-09-17T23:32:32.959525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:22.943613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"174'690","reported_seq":7987,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954629+0000","last_change":"2024-09-17T23:32:24.884398+0000","last_active":"2024-09-17T23:32:32.954629+0000","last_peered":"2024-09-17T23:32:32.954629+0000","last_clean":"2024-09-17T23:32:32.954629+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:32.954629+0000","last_undegraded":"2024-09-17T23:32:32.954629+0000","last_fullsized":"2024-09-17T23:32:32.954629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:18.501946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955603+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:32.955603+0000","last_peered":"2024-09-17T23:32:32.955603+0000","last_clean":"2024-09-17T23:32:32.955603+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:32.955603+0000","last_undegraded":"2024-09-17T23:32:32.955603+0000","last_fullsized":"2024-09-17T23:32:32.955603+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"172'699","reported_seq":12525,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963637+0000","last_change":"2024-09-17T23:32:32.963637+0000","last_active":"2024-09-17T23:32:32.963637+0000","last_peered":"2024-09-17T23:32:32.963637+0000","last_clean":"2024-09-17T23:32:32.963637+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:32.963637+0000","last_undegraded":"2024-09-17T23:32:32.963637+0000","last_fullsized":"2024-09-17T23:32:32.963637+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:06.933631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954229+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:32.954229+0000","last_peered":"2024-09-17T23:32:32.954229+0000","last_clean":"2024-09-17T23:32:32.954229+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:32.954229+0000","last_undegraded":"2024-09-17T23:32:32.954229+0000","last_fullsized":"2024-09-17T23:32:32.954229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"174'949","reported_seq":9718,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299168+0000","last_change":"2024-09-17T23:32:23.880388+0000","last_active":"2024-09-17T23:32:33.299168+0000","last_peered":"2024-09-17T23:32:33.299168+0000","last_clean":"2024-09-17T23:32:33.299168+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:33.299168+0000","last_undegraded":"2024-09-17T23:32:33.299168+0000","last_fullsized":"2024-09-17T23:32:33.299168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:19.966648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959403+0000","last_change":"2024-09-17T23:32:20.871165+0000","last_active":"2024-09-17T23:32:32.959403+0000","last_peered":"2024-09-17T23:32:32.959403+0000","last_clean":"2024-09-17T23:32:32.959403+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:32.959403+0000","last_undegraded":"2024-09-17T23:32:32.959403+0000","last_fullsized":"2024-09-17T23:32:32.959403+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"173'717","reported_seq":11659,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954280+0000","last_change":"2024-09-17T23:32:24.882213+0000","last_active":"2024-09-17T23:32:32.954280+0000","last_peered":"2024-09-17T23:32:32.954280+0000","last_clean":"2024-09-17T23:32:32.954280+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:32.954280+0000","last_undegraded":"2024-09-17T23:32:32.954280+0000","last_fullsized":"2024-09-17T23:32:32.954280+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:18.707223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":28,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959438+0000","last_change":"2024-09-17T23:32:24.887027+0000","last_active":"2024-09-17T23:32:32.959438+0000","last_peered":"2024-09-17T23:32:32.959438+0000","last_clean":"2024-09-17T23:32:32.959438+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:32.959438+0000","last_undegraded":"2024-09-17T23:32:32.959438+0000","last_fullsized":"2024-09-17T23:32:32.959438+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:50.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":454,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959142+0000","last_change":"2024-09-17T23:32:22.879076+0000","last_active":"2024-09-17T23:32:32.959142+0000","last_peered":"2024-09-17T23:32:32.959142+0000","last_clean":"2024-09-17T23:32:32.959142+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:32.959142+0000","last_undegraded":"2024-09-17T23:32:32.959142+0000","last_fullsized":"2024-09-17T23:32:32.959142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:01.370196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"174'491","reported_seq":15228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299131+0000","last_change":"2024-09-17T23:32:23.878098+0000","last_active":"2024-09-17T23:32:33.299131+0000","last_peered":"2024-09-17T23:32:33.299131+0000","last_clean":"2024-09-17T23:32:33.299131+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:33.299131+0000","last_undegraded":"2024-09-17T23:32:33.299131+0000","last_fullsized":"2024-09-17T23:32:33.299131+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.816169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"168'741","reported_seq":11750,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955122+0000","last_change":"2024-09-17T23:32:17.869242+0000","last_active":"2024-09-17T23:32:32.955122+0000","last_peered":"2024-09-17T23:32:32.955122+0000","last_clean":"2024-09-17T23:32:32.955122+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:32.955122+0000","last_undegraded":"2024-09-17T23:32:32.955122+0000","last_fullsized":"2024-09-17T23:32:32.955122+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:20.421701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'630","reported_seq":10164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955543+0000","last_change":"2024-09-17T23:32:26.875971+0000","last_active":"2024-09-17T23:32:32.955543+0000","last_peered":"2024-09-17T23:32:32.955543+0000","last_clean":"2024-09-17T23:32:32.955543+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:32.955543+0000","last_undegraded":"2024-09-17T23:32:32.955543+0000","last_fullsized":"2024-09-17T23:32:32.955543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:15.468503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"174'953","reported_seq":11755,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954268+0000","last_change":"2024-09-17T23:32:17.873346+0000","last_active":"2024-09-17T23:32:32.954268+0000","last_peered":"2024-09-17T23:32:32.954268+0000","last_clean":"2024-09-17T23:32:32.954268+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:32.954268+0000","last_undegraded":"2024-09-17T23:32:32.954268+0000","last_fullsized":"2024-09-17T23:32:32.954268+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:06:59.962287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'510","reported_seq":5624,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955422+0000","last_change":"2024-09-17T23:32:17.873091+0000","last_active":"2024-09-17T23:32:32.955422+0000","last_peered":"2024-09-17T23:32:32.955422+0000","last_clean":"2024-09-17T23:32:32.955422+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:32.955422+0000","last_undegraded":"2024-09-17T23:32:32.955422+0000","last_fullsized":"2024-09-17T23:32:32.955422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:28:21.353619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'436","reported_seq":9861,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959619+0000","last_change":"2024-09-17T23:32:22.878491+0000","last_active":"2024-09-17T23:32:32.959619+0000","last_peered":"2024-09-17T23:32:32.959619+0000","last_clean":"2024-09-17T23:32:32.959619+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:32.959619+0000","last_undegraded":"2024-09-17T23:32:32.959619+0000","last_fullsized":"2024-09-17T23:32:32.959619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:52.466160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"170'749","reported_seq":10350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959730+0000","last_change":"2024-09-17T23:32:26.876897+0000","last_active":"2024-09-17T23:32:32.959730+0000","last_peered":"2024-09-17T23:32:32.959730+0000","last_clean":"2024-09-17T23:32:32.959730+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:32.959730+0000","last_undegraded":"2024-09-17T23:32:32.959730+0000","last_fullsized":"2024-09-17T23:32:32.959730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:42.259419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"173'386","reported_seq":5130,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955226+0000","last_change":"2024-09-17T23:32:26.876451+0000","last_active":"2024-09-17T23:32:32.955226+0000","last_peered":"2024-09-17T23:32:32.955226+0000","last_clean":"2024-09-17T23:32:32.955226+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:32.955226+0000","last_undegraded":"2024-09-17T23:32:32.955226+0000","last_fullsized":"2024-09-17T23:32:32.955226+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:28.427580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"171'917","reported_seq":7998,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959251+0000","last_change":"2024-09-17T23:32:23.878627+0000","last_active":"2024-09-17T23:32:32.959251+0000","last_peered":"2024-09-17T23:32:32.959251+0000","last_clean":"2024-09-17T23:32:32.959251+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:32.959251+0000","last_undegraded":"2024-09-17T23:32:32.959251+0000","last_fullsized":"2024-09-17T23:32:32.959251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:13.949860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"173'380","reported_seq":8679,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955044+0000","last_change":"2024-09-17T23:32:17.871775+0000","last_active":"2024-09-17T23:32:32.955044+0000","last_peered":"2024-09-17T23:32:32.955044+0000","last_clean":"2024-09-17T23:32:32.955044+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:32.955044+0000","last_undegraded":"2024-09-17T23:32:32.955044+0000","last_fullsized":"2024-09-17T23:32:32.955044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:41.522918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"172'447","reported_seq":5925,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959067+0000","last_change":"2024-09-17T23:32:26.874708+0000","last_active":"2024-09-17T23:32:32.959067+0000","last_peered":"2024-09-17T23:32:32.959067+0000","last_clean":"2024-09-17T23:32:32.959067+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:32.959067+0000","last_undegraded":"2024-09-17T23:32:32.959067+0000","last_fullsized":"2024-09-17T23:32:32.959067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:23.493952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"174'605","reported_seq":15334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959350+0000","last_change":"2024-09-17T23:32:24.890401+0000","last_active":"2024-09-17T23:32:32.959350+0000","last_peered":"2024-09-17T23:32:32.959350+0000","last_clean":"2024-09-17T23:32:32.959350+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:32.959350+0000","last_undegraded":"2024-09-17T23:32:32.959350+0000","last_fullsized":"2024-09-17T23:32:32.959350+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:18.258055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19120,"ondisk_log_size":19120,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500691,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43140,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94328700,"statfs":{"total":96636764160,"available":96592588800,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640303,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51452,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320388,"statfs":{"total":96636764160,"available":96584077312,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640303,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77816,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640303,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37560,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334280,"statfs":{"total":96636764160,"available":96598302720,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2306048,"data_stored":1549280,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640303,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69508,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302332,"statfs":{"total":96636764160,"available":96565587968,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:37.834 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T23:32:37.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:38.126 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:38.127 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:38.138 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-17T23:32:36.399530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19240,"ondisk_log_size":19240,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":326808,"kb_used_data":13372,"kb_used_omap":27,"kb_used_meta":313252,"kb_avail":565904232,"statfs":{"total":579820584960,"available":579485933568,"internally_reserved":0,"allocated":13692928,"data_stored":6939227,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":320770952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001238"},"pg_stats":[{"pgid":"3.1f","version":"174'465","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963140+0000","last_change":"2024-09-17T23:32:32.963140+0000","last_active":"2024-09-17T23:32:32.963140+0000","last_peered":"2024-09-17T23:32:32.963140+0000","last_clean":"2024-09-17T23:32:32.963140+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:32.963140+0000","last_undegraded":"2024-09-17T23:32:32.963140+0000","last_fullsized":"2024-09-17T23:32:32.963140+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:04.457427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"174'578","reported_seq":9351,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954468+0000","last_change":"2024-09-17T23:32:22.877385+0000","last_active":"2024-09-17T23:32:32.954468+0000","last_peered":"2024-09-17T23:32:32.954468+0000","last_clean":"2024-09-17T23:32:32.954468+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:32.954468+0000","last_undegraded":"2024-09-17T23:32:32.954468+0000","last_fullsized":"2024-09-17T23:32:32.954468+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:09.418470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"174'373","reported_seq":5985,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959626+0000","last_change":"2024-09-17T23:32:26.875330+0000","last_active":"2024-09-17T23:32:32.959626+0000","last_peered":"2024-09-17T23:32:32.959626+0000","last_clean":"2024-09-17T23:32:32.959626+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:32.959626+0000","last_undegraded":"2024-09-17T23:32:32.959626+0000","last_fullsized":"2024-09-17T23:32:32.959626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:18.413939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"174'471","reported_seq":9402,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960398+0000","last_change":"2024-09-17T23:32:24.889849+0000","last_active":"2024-09-17T23:32:32.960398+0000","last_peered":"2024-09-17T23:32:32.960398+0000","last_clean":"2024-09-17T23:32:32.960398+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:32.960398+0000","last_undegraded":"2024-09-17T23:32:32.960398+0000","last_fullsized":"2024-09-17T23:32:32.960398+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:36.156958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"173'452","reported_seq":19741,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954200+0000","last_change":"2024-09-17T23:32:17.870826+0000","last_active":"2024-09-17T23:32:32.954200+0000","last_peered":"2024-09-17T23:32:32.954200+0000","last_clean":"2024-09-17T23:32:32.954200+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:32.954200+0000","last_undegraded":"2024-09-17T23:32:32.954200+0000","last_fullsized":"2024-09-17T23:32:32.954200+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:57.276383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"169'882","reported_seq":7762,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960630+0000","last_change":"2024-09-17T23:32:22.881407+0000","last_active":"2024-09-17T23:32:32.960630+0000","last_peered":"2024-09-17T23:32:32.960630+0000","last_clean":"2024-09-17T23:32:32.960630+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:32.960630+0000","last_undegraded":"2024-09-17T23:32:32.960630+0000","last_fullsized":"2024-09-17T23:32:32.960630+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:41.917085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"173'844","reported_seq":6806,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299237+0000","last_change":"2024-09-17T23:32:17.877448+0000","last_active":"2024-09-17T23:32:33.299237+0000","last_peered":"2024-09-17T23:32:33.299237+0000","last_clean":"2024-09-17T23:32:33.299237+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:33.299237+0000","last_undegraded":"2024-09-17T23:32:33.299237+0000","last_fullsized":"2024-09-17T23:32:33.299237+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:25.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"170'487","reported_seq":4566,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954121+0000","last_change":"2024-09-17T23:32:17.869351+0000","last_active":"2024-09-17T23:32:32.954121+0000","last_peered":"2024-09-17T23:32:32.954121+0000","last_clean":"2024-09-17T23:32:32.954121+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:32.954121+0000","last_undegraded":"2024-09-17T23:32:32.954121+0000","last_fullsized":"2024-09-17T23:32:32.954121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:48.297102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"173'675","reported_seq":8334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955935+0000","last_change":"2024-09-17T23:32:17.874867+0000","last_active":"2024-09-17T23:32:32.955935+0000","last_peered":"2024-09-17T23:32:32.955935+0000","last_clean":"2024-09-17T23:32:32.955935+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:32.955935+0000","last_undegraded":"2024-09-17T23:32:32.955935+0000","last_fullsized":"2024-09-17T23:32:32.955935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:31.383792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"173'394","reported_seq":4573,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959820+0000","last_change":"2024-09-17T23:32:26.872510+0000","last_active":"2024-09-17T23:32:32.959820+0000","last_peered":"2024-09-17T23:32:32.959820+0000","last_clean":"2024-09-17T23:32:32.959820+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:32.959820+0000","last_undegraded":"2024-09-17T23:32:32.959820+0000","last_fullsized":"2024-09-17T23:32:32.959820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:36.272282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"174'635","reported_seq":13063,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954948+0000","last_change":"2024-09-17T23:32:17.872364+0000","last_active":"2024-09-17T23:32:32.954948+0000","last_peered":"2024-09-17T23:32:32.954948+0000","last_clean":"2024-09-17T23:32:32.954948+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:32.954948+0000","last_undegraded":"2024-09-17T23:32:32.954948+0000","last_fullsized":"2024-09-17T23:32:32.954948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:33.326848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'630","reported_seq":8364,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960280+0000","last_change":"2024-09-17T23:32:26.873492+0000","last_active":"2024-09-17T23:32:32.960280+0000","last_peered":"2024-09-17T23:32:32.960280+0000","last_clean":"2024-09-17T23:32:32.960280+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:32.960280+0000","last_undegraded":"2024-09-17T23:32:32.960280+0000","last_fullsized":"2024-09-17T23:32:32.960280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:01.374405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960092+0000","last_change":"2024-09-17T23:32:24.890813+0000","last_active":"2024-09-17T23:32:32.960092+0000","last_peered":"2024-09-17T23:32:32.960092+0000","last_clean":"2024-09-17T23:32:32.960092+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:32.960092+0000","last_undegraded":"2024-09-17T23:32:32.960092+0000","last_fullsized":"2024-09-17T23:32:32.960092+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:15.170449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"174'664","reported_seq":5270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299239+0000","last_change":"2024-09-17T23:32:17.874681+0000","last_active":"2024-09-17T23:32:33.299239+0000","last_peered":"2024-09-17T23:32:33.299239+0000","last_clean":"2024-09-17T23:32:33.299239+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:33.299239+0000","last_undegraded":"2024-09-17T23:32:33.299239+0000","last_fullsized":"2024-09-17T23:32:33.299239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:10.119882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299322+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:33.299322+0000","last_peered":"2024-09-17T23:32:33.299322+0000","last_clean":"2024-09-17T23:32:33.299322+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:33.299322+0000","last_undegraded":"2024-09-17T23:32:33.299322+0000","last_fullsized":"2024-09-17T23:32:33.299322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'453","reported_seq":12753,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299389+0000","last_change":"2024-09-17T23:32:17.875953+0000","last_active":"2024-09-17T23:32:33.299389+0000","last_peered":"2024-09-17T23:32:33.299389+0000","last_clean":"2024-09-17T23:32:33.299389+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:33.299389+0000","last_undegraded":"2024-09-17T23:32:33.299389+0000","last_fullsized":"2024-09-17T23:32:33.299389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:08.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959531+0000","last_change":"2024-09-17T23:32:24.883350+0000","last_active":"2024-09-17T23:32:32.959531+0000","last_peered":"2024-09-17T23:32:32.959531+0000","last_clean":"2024-09-17T23:32:32.959531+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:32.959531+0000","last_undegraded":"2024-09-17T23:32:32.959531+0000","last_fullsized":"2024-09-17T23:32:32.959531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:05.948442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"174'416","reported_seq":7427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299244+0000","last_change":"2024-09-17T23:32:17.875596+0000","last_active":"2024-09-17T23:32:33.299244+0000","last_peered":"2024-09-17T23:32:33.299244+0000","last_clean":"2024-09-17T23:32:33.299244+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:33.299244+0000","last_undegraded":"2024-09-17T23:32:33.299244+0000","last_fullsized":"2024-09-17T23:32:33.299244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:19.550098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.958724+0000","last_change":"2024-09-17T23:32:32.958724+0000","last_active":"2024-09-17T23:32:32.958724+0000","last_peered":"2024-09-17T23:32:32.958724+0000","last_clean":"2024-09-17T23:32:32.958724+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:32.958724+0000","last_undegraded":"2024-09-17T23:32:32.958724+0000","last_fullsized":"2024-09-17T23:32:32.958724+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:25.311067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"171'401","reported_seq":10668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959525+0000","last_change":"2024-09-17T23:32:26.872700+0000","last_active":"2024-09-17T23:32:32.959525+0000","last_peered":"2024-09-17T23:32:32.959525+0000","last_clean":"2024-09-17T23:32:32.959525+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:32.959525+0000","last_undegraded":"2024-09-17T23:32:32.959525+0000","last_fullsized":"2024-09-17T23:32:32.959525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:22.943613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"174'690","reported_seq":7987,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954629+0000","last_change":"2024-09-17T23:32:24.884398+0000","last_active":"2024-09-17T23:32:32.954629+0000","last_peered":"2024-09-17T23:32:32.954629+0000","last_clean":"2024-09-17T23:32:32.954629+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:32.954629+0000","last_undegraded":"2024-09-17T23:32:32.954629+0000","last_fullsized":"2024-09-17T23:32:32.954629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:18.501946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955603+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:32.955603+0000","last_peered":"2024-09-17T23:32:32.955603+0000","last_clean":"2024-09-17T23:32:32.955603+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:32.955603+0000","last_undegraded":"2024-09-17T23:32:32.955603+0000","last_fullsized":"2024-09-17T23:32:32.955603+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"172'699","reported_seq":12525,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963637+0000","last_change":"2024-09-17T23:32:32.963637+0000","last_active":"2024-09-17T23:32:32.963637+0000","last_peered":"2024-09-17T23:32:32.963637+0000","last_clean":"2024-09-17T23:32:32.963637+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:32.963637+0000","last_undegraded":"2024-09-17T23:32:32.963637+0000","last_fullsized":"2024-09-17T23:32:32.963637+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:06.933631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954229+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:32.954229+0000","last_peered":"2024-09-17T23:32:32.954229+0000","last_clean":"2024-09-17T23:32:32.954229+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:32.954229+0000","last_undegraded":"2024-09-17T23:32:32.954229+0000","last_fullsized":"2024-09-17T23:32:32.954229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"174'949","reported_seq":9718,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299168+0000","last_change":"2024-09-17T23:32:23.880388+0000","last_active":"2024-09-17T23:32:33.299168+0000","last_peered":"2024-09-17T23:32:33.299168+0000","last_clean":"2024-09-17T23:32:33.299168+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:33.299168+0000","last_undegraded":"2024-09-17T23:32:33.299168+0000","last_fullsized":"2024-09-17T23:32:33.299168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:19.966648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959403+0000","last_change":"2024-09-17T23:32:20.871165+0000","last_active":"2024-09-17T23:32:32.959403+0000","last_peered":"2024-09-17T23:32:32.959403+0000","last_clean":"2024-09-17T23:32:32.959403+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:32.959403+0000","last_undegraded":"2024-09-17T23:32:32.959403+0000","last_fullsized":"2024-09-17T23:32:32.959403+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"173'717","reported_seq":11659,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954280+0000","last_change":"2024-09-17T23:32:24.882213+0000","last_active":"2024-09-17T23:32:32.954280+0000","last_peered":"2024-09-17T23:32:32.954280+0000","last_clean":"2024-09-17T23:32:32.954280+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:32.954280+0000","last_undegraded":"2024-09-17T23:32:32.954280+0000","last_fullsized":"2024-09-17T23:32:32.954280+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:18.707223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":28,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959438+0000","last_change":"2024-09-17T23:32:24.887027+0000","last_active":"2024-09-17T23:32:32.959438+0000","last_peered":"2024-09-17T23:32:32.959438+0000","last_clean":"2024-09-17T23:32:32.959438+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:32.959438+0000","last_undegraded":"2024-09-17T23:32:32.959438+0000","last_fullsized":"2024-09-17T23:32:32.959438+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:50.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":454,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959142+0000","last_change":"2024-09-17T23:32:22.879076+0000","last_active":"2024-09-17T23:32:32.959142+0000","last_peered":"2024-09-17T23:32:32.959142+0000","last_clean":"2024-09-17T23:32:32.959142+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:32.959142+0000","last_undegraded":"2024-09-17T23:32:32.959142+0000","last_fullsized":"2024-09-17T23:32:32.959142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:01.370196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"174'491","reported_seq":15228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299131+0000","last_change":"2024-09-17T23:32:23.878098+0000","last_active":"2024-09-17T23:32:33.299131+0000","last_peered":"2024-09-17T23:32:33.299131+0000","last_clean":"2024-09-17T23:32:33.299131+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:33.299131+0000","last_undegraded":"2024-09-17T23:32:33.299131+0000","last_fullsized":"2024-09-17T23:32:33.299131+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.816169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"168'741","reported_seq":11750,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955122+0000","last_change":"2024-09-17T23:32:17.869242+0000","last_active":"2024-09-17T23:32:32.955122+0000","last_peered":"2024-09-17T23:32:32.955122+0000","last_clean":"2024-09-17T23:32:32.955122+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:32.955122+0000","last_undegraded":"2024-09-17T23:32:32.955122+0000","last_fullsized":"2024-09-17T23:32:32.955122+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:20.421701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'630","reported_seq":10164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955543+0000","last_change":"2024-09-17T23:32:26.875971+0000","last_active":"2024-09-17T23:32:32.955543+0000","last_peered":"2024-09-17T23:32:32.955543+0000","last_clean":"2024-09-17T23:32:32.955543+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:32.955543+0000","last_undegraded":"2024-09-17T23:32:32.955543+0000","last_fullsized":"2024-09-17T23:32:32.955543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:15.468503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"174'953","reported_seq":11755,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954268+0000","last_change":"2024-09-17T23:32:17.873346+0000","last_active":"2024-09-17T23:32:32.954268+0000","last_peered":"2024-09-17T23:32:32.954268+0000","last_clean":"2024-09-17T23:32:32.954268+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:32.954268+0000","last_undegraded":"2024-09-17T23:32:32.954268+0000","last_fullsized":"2024-09-17T23:32:32.954268+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:06:59.962287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'510","reported_seq":5624,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955422+0000","last_change":"2024-09-17T23:32:17.873091+0000","last_active":"2024-09-17T23:32:32.955422+0000","last_peered":"2024-09-17T23:32:32.955422+0000","last_clean":"2024-09-17T23:32:32.955422+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:32.955422+0000","last_undegraded":"2024-09-17T23:32:32.955422+0000","last_fullsized":"2024-09-17T23:32:32.955422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:28:21.353619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'436","reported_seq":9861,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959619+0000","last_change":"2024-09-17T23:32:22.878491+0000","last_active":"2024-09-17T23:32:32.959619+0000","last_peered":"2024-09-17T23:32:32.959619+0000","last_clean":"2024-09-17T23:32:32.959619+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:32.959619+0000","last_undegraded":"2024-09-17T23:32:32.959619+0000","last_fullsized":"2024-09-17T23:32:32.959619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:52.466160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"170'749","reported_seq":10350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959730+0000","last_change":"2024-09-17T23:32:26.876897+0000","last_active":"2024-09-17T23:32:32.959730+0000","last_peered":"2024-09-17T23:32:32.959730+0000","last_clean":"2024-09-17T23:32:32.959730+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:32.959730+0000","last_undegraded":"2024-09-17T23:32:32.959730+0000","last_fullsized":"2024-09-17T23:32:32.959730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:42.259419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"173'386","reported_seq":5130,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955226+0000","last_change":"2024-09-17T23:32:26.876451+0000","last_active":"2024-09-17T23:32:32.955226+0000","last_peered":"2024-09-17T23:32:32.955226+0000","last_clean":"2024-09-17T23:32:32.955226+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:32.955226+0000","last_undegraded":"2024-09-17T23:32:32.955226+0000","last_fullsized":"2024-09-17T23:32:32.955226+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:28.427580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"171'917","reported_seq":7998,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959251+0000","last_change":"2024-09-17T23:32:23.878627+0000","last_active":"2024-09-17T23:32:32.959251+0000","last_peered":"2024-09-17T23:32:32.959251+0000","last_clean":"2024-09-17T23:32:32.959251+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:32.959251+0000","last_undegraded":"2024-09-17T23:32:32.959251+0000","last_fullsized":"2024-09-17T23:32:32.959251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:13.949860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"173'380","reported_seq":8679,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955044+0000","last_change":"2024-09-17T23:32:17.871775+0000","last_active":"2024-09-17T23:32:32.955044+0000","last_peered":"2024-09-17T23:32:32.955044+0000","last_clean":"2024-09-17T23:32:32.955044+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:32.955044+0000","last_undegraded":"2024-09-17T23:32:32.955044+0000","last_fullsized":"2024-09-17T23:32:32.955044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:41.522918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"172'447","reported_seq":5925,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959067+0000","last_change":"2024-09-17T23:32:26.874708+0000","last_active":"2024-09-17T23:32:32.959067+0000","last_peered":"2024-09-17T23:32:32.959067+0000","last_clean":"2024-09-17T23:32:32.959067+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:32.959067+0000","last_undegraded":"2024-09-17T23:32:32.959067+0000","last_fullsized":"2024-09-17T23:32:32.959067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:23.493952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"174'605","reported_seq":15334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959350+0000","last_change":"2024-09-17T23:32:24.890401+0000","last_active":"2024-09-17T23:32:32.959350+0000","last_peered":"2024-09-17T23:32:32.959350+0000","last_clean":"2024-09-17T23:32:32.959350+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:32.959350+0000","last_undegraded":"2024-09-17T23:32:32.959350+0000","last_fullsized":"2024-09-17T23:32:32.959350+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:18.258055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19120,"ondisk_log_size":19120,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500691,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43140,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94328700,"statfs":{"total":96636764160,"available":96592588800,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640303,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51452,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320388,"statfs":{"total":96636764160,"available":96584077312,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640303,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77816,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294024,"statfs":{"total":96636764160,"available":96557080576,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640303,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37560,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334280,"statfs":{"total":96636764160,"available":96598302720,"internally_reserved":0,"allocated":2260992,"data_stored":958893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2252,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2306048,"data_stored":1549280,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640303,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69508,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302332,"statfs":{"total":96636764160,"available":96565587968,"internally_reserved":0,"allocated":2265088,"data_stored":958912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:38.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T23:32:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:38.429 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":189,"fsid":"b662004c-f841-43da-8884-84d1805b0cbb","created":"2024-09-17T23:28:46.056204+0000","modified":"2024-09-17T23:32:32.946757+0000","last_up_change":"2024-09-17T23:31:11.038388+0000","last_in_change":"2024-09-17T23:32:19.868974+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T23:28:58.406361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T23:29:03.876212+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T23:29:14.532518+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"174","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":174,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2c921380-aff6-4e43-bc78-48a393ed3673","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6817","nonce":2793055955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6819","nonce":2793055955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6823","nonce":2793055955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2793055955},{"type":"v1","addr":"172.21.15.31:6821","nonce":2793055955}]},"public_addr":"172.21.15.31:6817/2793055955","cluster_addr":"172.21.15.31:6819/2793055955","heartbeat_back_addr":"172.21.15.31:6823/2793055955","heartbeat_front_addr":"172.21.15.31:6821/2793055955","state":["exists","up"]},{"osd":1,"uuid":"c95ad2cb-7bfe-4d1a-8280-640a59b6e73e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6809","nonce":2784091455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6811","nonce":2784091455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6815","nonce":2784091455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2784091455},{"type":"v1","addr":"172.21.15.31:6813","nonce":2784091455}]},"public_addr":"172.21.15.31:6809/2784091455","cluster_addr":"172.21.15.31:6811/2784091455","heartbeat_back_addr":"172.21.15.31:6815/2784091455","heartbeat_front_addr":"172.21.15.31:6813/2784091455","state":["exists","up"]},{"osd":2,"uuid":"c7d869ca-7f8b-46df-882e-57da88ef6eda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6801","nonce":464841260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6803","nonce":464841260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6807","nonce":464841260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":464841260},{"type":"v1","addr":"172.21.15.31:6805","nonce":464841260}]},"public_addr":"172.21.15.31:6801/464841260","cluster_addr":"172.21.15.31:6803/464841260","heartbeat_back_addr":"172.21.15.31:6807/464841260","heartbeat_front_addr":"172.21.15.31:6805/464841260","state":["exists","up"]},{"osd":3,"uuid":"ef4b1c62-ae14-4e5a-9914-d674364f3051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6801","nonce":1436186932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6803","nonce":1436186932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6807","nonce":1436186932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1436186932},{"type":"v1","addr":"172.21.15.37:6805","nonce":1436186932}]},"public_addr":"172.21.15.37:6801/1436186932","cluster_addr":"172.21.15.37:6803/1436186932","heartbeat_back_addr":"172.21.15.37:6807/1436186932","heartbeat_front_addr":"172.21.15.37:6805/1436186932","state":["exists","up"]},{"osd":4,"uuid":"fcd61898-fc45-4d4a-83b2-4f1bf9720309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6817","nonce":154081689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6819","nonce":154081689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6823","nonce":154081689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":154081689},{"type":"v1","addr":"172.21.15.37:6821","nonce":154081689}]},"public_addr":"172.21.15.37:6817/154081689","cluster_addr":"172.21.15.37:6819/154081689","heartbeat_back_addr":"172.21.15.37:6823/154081689","heartbeat_front_addr":"172.21.15.37:6821/154081689","state":["exists","up"]},{"osd":5,"uuid":"1846216e-c369-4918-88be-300c9ee5397f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":93,"up_from":107,"up_thru":188,"down_at":94,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6809","nonce":1359318569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6811","nonce":1359318569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6815","nonce":1359318569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1359318569},{"type":"v1","addr":"172.21.15.37:6813","nonce":1359318569}]},"public_addr":"172.21.15.37:6809/1359318569","cluster_addr":"172.21.15.37:6811/1359318569","heartbeat_back_addr":"172.21.15.37:6815/1359318569","heartbeat_front_addr":"172.21.15.37:6813/1359318569","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:55.062494+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.641591+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.788407+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.360450+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.895512+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T23:28:54.797662+0000","dead_epoch":94}],"pg_upmap":[{"pgid":"3.11","osds":[0,4]},{"pgid":"3.12","osds":[2,1]},{"pgid":"3.17","osds":[5,4]}],"pg_upmap_items":[{"pgid":"3.12","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T23:32:39.442 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T23:32:39.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:32:39.583 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:32:39.583 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:39.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T23:32:39.883 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T23:32:39.897 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T23:32:39.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:32:40.023 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:40.024 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:32:40.024 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:40.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T23:32:40.324 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T23:32:40.337 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T23:32:40.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:32:40.464 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:40.464 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:32:40.464 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:40.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T23:32:40.764 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T23:32:40.777 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T23:32:40.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:32:40.904 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:40.904 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:32:40.904 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:40.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T23:32:41.209 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T23:32:41.221 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T23:32:41.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:32:41.347 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:41.347 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:32:41.347 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:41.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T23:32:41.649 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T23:32:41.662 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T23:32:41.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T23:32:41.787 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-17T23:32:41.788 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T23:32:41.788 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-17T23:32:41.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T23:32:42.093 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T23:32:42.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:32:42.394 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:32:42.395 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:32:42.406 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-17T23:32:40.400362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19240,"ondisk_log_size":19240,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":326872,"kb_used_data":13432,"kb_used_omap":27,"kb_used_meta":313252,"kb_avail":565904168,"statfs":{"total":579820584960,"available":579485868032,"internally_reserved":0,"allocated":13754368,"data_stored":6964512,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":320770952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.456975"},"pg_stats":[{"pgid":"3.1f","version":"174'465","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963140+0000","last_change":"2024-09-17T23:32:32.963140+0000","last_active":"2024-09-17T23:32:32.963140+0000","last_peered":"2024-09-17T23:32:32.963140+0000","last_clean":"2024-09-17T23:32:32.963140+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:32.963140+0000","last_undegraded":"2024-09-17T23:32:32.963140+0000","last_fullsized":"2024-09-17T23:32:32.963140+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:04.457427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"174'578","reported_seq":9351,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954468+0000","last_change":"2024-09-17T23:32:22.877385+0000","last_active":"2024-09-17T23:32:32.954468+0000","last_peered":"2024-09-17T23:32:32.954468+0000","last_clean":"2024-09-17T23:32:32.954468+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:32.954468+0000","last_undegraded":"2024-09-17T23:32:32.954468+0000","last_fullsized":"2024-09-17T23:32:32.954468+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:08:09.418470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"174'373","reported_seq":5985,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959626+0000","last_change":"2024-09-17T23:32:26.875330+0000","last_active":"2024-09-17T23:32:32.959626+0000","last_peered":"2024-09-17T23:32:32.959626+0000","last_clean":"2024-09-17T23:32:32.959626+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:32.959626+0000","last_undegraded":"2024-09-17T23:32:32.959626+0000","last_fullsized":"2024-09-17T23:32:32.959626+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:00:18.413939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"174'471","reported_seq":9402,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960398+0000","last_change":"2024-09-17T23:32:24.889849+0000","last_active":"2024-09-17T23:32:32.960398+0000","last_peered":"2024-09-17T23:32:32.960398+0000","last_clean":"2024-09-17T23:32:32.960398+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:32.960398+0000","last_undegraded":"2024-09-17T23:32:32.960398+0000","last_fullsized":"2024-09-17T23:32:32.960398+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:00:36.156958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"173'452","reported_seq":19741,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954200+0000","last_change":"2024-09-17T23:32:17.870826+0000","last_active":"2024-09-17T23:32:32.954200+0000","last_peered":"2024-09-17T23:32:32.954200+0000","last_clean":"2024-09-17T23:32:32.954200+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:32.954200+0000","last_undegraded":"2024-09-17T23:32:32.954200+0000","last_fullsized":"2024-09-17T23:32:32.954200+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:19:57.276383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"169'882","reported_seq":7762,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960630+0000","last_change":"2024-09-17T23:32:22.881407+0000","last_active":"2024-09-17T23:32:32.960630+0000","last_peered":"2024-09-17T23:32:32.960630+0000","last_clean":"2024-09-17T23:32:32.960630+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:32.960630+0000","last_undegraded":"2024-09-17T23:32:32.960630+0000","last_fullsized":"2024-09-17T23:32:32.960630+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:00:41.917085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"173'844","reported_seq":6806,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299237+0000","last_change":"2024-09-17T23:32:17.877448+0000","last_active":"2024-09-17T23:32:33.299237+0000","last_peered":"2024-09-17T23:32:33.299237+0000","last_clean":"2024-09-17T23:32:33.299237+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:33.299237+0000","last_undegraded":"2024-09-17T23:32:33.299237+0000","last_fullsized":"2024-09-17T23:32:33.299237+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:25.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"170'487","reported_seq":4566,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954121+0000","last_change":"2024-09-17T23:32:17.869351+0000","last_active":"2024-09-17T23:32:32.954121+0000","last_peered":"2024-09-17T23:32:32.954121+0000","last_clean":"2024-09-17T23:32:32.954121+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:32.954121+0000","last_undegraded":"2024-09-17T23:32:32.954121+0000","last_fullsized":"2024-09-17T23:32:32.954121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:48.297102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"173'675","reported_seq":8334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955935+0000","last_change":"2024-09-17T23:32:17.874867+0000","last_active":"2024-09-17T23:32:32.955935+0000","last_peered":"2024-09-17T23:32:32.955935+0000","last_clean":"2024-09-17T23:32:32.955935+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:32.955935+0000","last_undegraded":"2024-09-17T23:32:32.955935+0000","last_fullsized":"2024-09-17T23:32:32.955935+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:31.383792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"173'394","reported_seq":4573,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959820+0000","last_change":"2024-09-17T23:32:26.872510+0000","last_active":"2024-09-17T23:32:32.959820+0000","last_peered":"2024-09-17T23:32:32.959820+0000","last_clean":"2024-09-17T23:32:32.959820+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:32.959820+0000","last_undegraded":"2024-09-17T23:32:32.959820+0000","last_fullsized":"2024-09-17T23:32:32.959820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:36.272282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"174'635","reported_seq":13063,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954948+0000","last_change":"2024-09-17T23:32:17.872364+0000","last_active":"2024-09-17T23:32:32.954948+0000","last_peered":"2024-09-17T23:32:32.954948+0000","last_clean":"2024-09-17T23:32:32.954948+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:32.954948+0000","last_undegraded":"2024-09-17T23:32:32.954948+0000","last_fullsized":"2024-09-17T23:32:32.954948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:33.326848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"171'630","reported_seq":8364,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960280+0000","last_change":"2024-09-17T23:32:26.873492+0000","last_active":"2024-09-17T23:32:32.960280+0000","last_peered":"2024-09-17T23:32:32.960280+0000","last_clean":"2024-09-17T23:32:32.960280+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:32.960280+0000","last_undegraded":"2024-09-17T23:32:32.960280+0000","last_fullsized":"2024-09-17T23:32:32.960280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:01.374405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.960092+0000","last_change":"2024-09-17T23:32:24.890813+0000","last_active":"2024-09-17T23:32:32.960092+0000","last_peered":"2024-09-17T23:32:32.960092+0000","last_clean":"2024-09-17T23:32:32.960092+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:32.960092+0000","last_undegraded":"2024-09-17T23:32:32.960092+0000","last_fullsized":"2024-09-17T23:32:32.960092+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:03:15.170449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"174'664","reported_seq":5270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299239+0000","last_change":"2024-09-17T23:32:17.874681+0000","last_active":"2024-09-17T23:32:33.299239+0000","last_peered":"2024-09-17T23:32:33.299239+0000","last_clean":"2024-09-17T23:32:33.299239+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:33.299239+0000","last_undegraded":"2024-09-17T23:32:33.299239+0000","last_fullsized":"2024-09-17T23:32:33.299239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:10.119882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299322+0000","last_change":"2024-09-17T23:29:07.501252+0000","last_active":"2024-09-17T23:32:33.299322+0000","last_peered":"2024-09-17T23:32:33.299322+0000","last_clean":"2024-09-17T23:32:33.299322+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:33.299322+0000","last_undegraded":"2024-09-17T23:32:33.299322+0000","last_fullsized":"2024-09-17T23:32:33.299322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:56.355939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"171'453","reported_seq":12753,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299389+0000","last_change":"2024-09-17T23:32:17.875953+0000","last_active":"2024-09-17T23:32:33.299389+0000","last_peered":"2024-09-17T23:32:33.299389+0000","last_clean":"2024-09-17T23:32:33.299389+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:33.299389+0000","last_undegraded":"2024-09-17T23:32:33.299389+0000","last_fullsized":"2024-09-17T23:32:33.299389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:02:08.301975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":321,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959531+0000","last_change":"2024-09-17T23:32:24.883350+0000","last_active":"2024-09-17T23:32:32.959531+0000","last_peered":"2024-09-17T23:32:32.959531+0000","last_clean":"2024-09-17T23:32:32.959531+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:32.959531+0000","last_undegraded":"2024-09-17T23:32:32.959531+0000","last_fullsized":"2024-09-17T23:32:32.959531+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:10:05.948442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"174'416","reported_seq":7427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299244+0000","last_change":"2024-09-17T23:32:17.875596+0000","last_active":"2024-09-17T23:32:33.299244+0000","last_peered":"2024-09-17T23:32:33.299244+0000","last_clean":"2024-09-17T23:32:33.299244+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:33.299244+0000","last_undegraded":"2024-09-17T23:32:33.299244+0000","last_fullsized":"2024-09-17T23:32:33.299244+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:19.550098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.958724+0000","last_change":"2024-09-17T23:32:32.958724+0000","last_active":"2024-09-17T23:32:32.958724+0000","last_peered":"2024-09-17T23:32:32.958724+0000","last_clean":"2024-09-17T23:32:32.958724+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:32.958724+0000","last_undegraded":"2024-09-17T23:32:32.958724+0000","last_fullsized":"2024-09-17T23:32:32.958724+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:25.311067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"171'401","reported_seq":10668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959525+0000","last_change":"2024-09-17T23:32:26.872700+0000","last_active":"2024-09-17T23:32:32.959525+0000","last_peered":"2024-09-17T23:32:32.959525+0000","last_clean":"2024-09-17T23:32:32.959525+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:32.959525+0000","last_undegraded":"2024-09-17T23:32:32.959525+0000","last_fullsized":"2024-09-17T23:32:32.959525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:22.943613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"174'690","reported_seq":7987,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954629+0000","last_change":"2024-09-17T23:32:24.884398+0000","last_active":"2024-09-17T23:32:32.954629+0000","last_peered":"2024-09-17T23:32:32.954629+0000","last_clean":"2024-09-17T23:32:32.954629+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:32.954629+0000","last_undegraded":"2024-09-17T23:32:32.954629+0000","last_fullsized":"2024-09-17T23:32:32.954629+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:18.501946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955603+0000","last_change":"2024-09-17T23:31:55.016217+0000","last_active":"2024-09-17T23:32:32.955603+0000","last_peered":"2024-09-17T23:32:32.955603+0000","last_clean":"2024-09-17T23:32:32.955603+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:32.955603+0000","last_undegraded":"2024-09-17T23:32:32.955603+0000","last_fullsized":"2024-09-17T23:32:32.955603+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:46:23.454486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"172'699","reported_seq":12525,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.963637+0000","last_change":"2024-09-17T23:32:32.963637+0000","last_active":"2024-09-17T23:32:32.963637+0000","last_peered":"2024-09-17T23:32:32.963637+0000","last_clean":"2024-09-17T23:32:32.963637+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:32.963637+0000","last_undegraded":"2024-09-17T23:32:32.963637+0000","last_fullsized":"2024-09-17T23:32:32.963637+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:06.933631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954229+0000","last_change":"2024-09-17T23:31:12.052191+0000","last_active":"2024-09-17T23:32:32.954229+0000","last_peered":"2024-09-17T23:32:32.954229+0000","last_clean":"2024-09-17T23:32:32.954229+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:32.954229+0000","last_undegraded":"2024-09-17T23:32:32.954229+0000","last_fullsized":"2024-09-17T23:32:32.954229+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:31.346226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"174'949","reported_seq":9718,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299168+0000","last_change":"2024-09-17T23:32:23.880388+0000","last_active":"2024-09-17T23:32:33.299168+0000","last_peered":"2024-09-17T23:32:33.299168+0000","last_clean":"2024-09-17T23:32:33.299168+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:33.299168+0000","last_undegraded":"2024-09-17T23:32:33.299168+0000","last_fullsized":"2024-09-17T23:32:33.299168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:19.966648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959403+0000","last_change":"2024-09-17T23:32:20.871165+0000","last_active":"2024-09-17T23:32:32.959403+0000","last_peered":"2024-09-17T23:32:32.959403+0000","last_clean":"2024-09-17T23:32:32.959403+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:32.959403+0000","last_undegraded":"2024-09-17T23:32:32.959403+0000","last_fullsized":"2024-09-17T23:32:32.959403+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:16:36.586940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"173'717","reported_seq":11659,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954280+0000","last_change":"2024-09-17T23:32:24.882213+0000","last_active":"2024-09-17T23:32:32.954280+0000","last_peered":"2024-09-17T23:32:32.954280+0000","last_clean":"2024-09-17T23:32:32.954280+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:32.954280+0000","last_undegraded":"2024-09-17T23:32:32.954280+0000","last_fullsized":"2024-09-17T23:32:32.954280+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:18.707223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":28,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959438+0000","last_change":"2024-09-17T23:32:24.887027+0000","last_active":"2024-09-17T23:32:32.959438+0000","last_peered":"2024-09-17T23:32:32.959438+0000","last_clean":"2024-09-17T23:32:32.959438+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:32.959438+0000","last_undegraded":"2024-09-17T23:32:32.959438+0000","last_fullsized":"2024-09-17T23:32:32.959438+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:04.482750+0000","last_clean_scrub_stamp":"2024-09-17T23:29:04.482750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:22:50.075968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":454,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959142+0000","last_change":"2024-09-17T23:32:22.879076+0000","last_active":"2024-09-17T23:32:32.959142+0000","last_peered":"2024-09-17T23:32:32.959142+0000","last_clean":"2024-09-17T23:32:32.959142+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:32.959142+0000","last_undegraded":"2024-09-17T23:32:32.959142+0000","last_fullsized":"2024-09-17T23:32:32.959142+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:28:58.457979+0000","last_clean_scrub_stamp":"2024-09-17T23:28:58.457979+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:01.370196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"174'491","reported_seq":15228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:33.299131+0000","last_change":"2024-09-17T23:32:23.878098+0000","last_active":"2024-09-17T23:32:33.299131+0000","last_peered":"2024-09-17T23:32:33.299131+0000","last_clean":"2024-09-17T23:32:33.299131+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:33.299131+0000","last_undegraded":"2024-09-17T23:32:33.299131+0000","last_fullsized":"2024-09-17T23:32:33.299131+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:33.816169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"168'741","reported_seq":11750,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955122+0000","last_change":"2024-09-17T23:32:17.869242+0000","last_active":"2024-09-17T23:32:32.955122+0000","last_peered":"2024-09-17T23:32:32.955122+0000","last_clean":"2024-09-17T23:32:32.955122+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:32.955122+0000","last_undegraded":"2024-09-17T23:32:32.955122+0000","last_fullsized":"2024-09-17T23:32:32.955122+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:20.421701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"171'630","reported_seq":10164,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955543+0000","last_change":"2024-09-17T23:32:26.875971+0000","last_active":"2024-09-17T23:32:32.955543+0000","last_peered":"2024-09-17T23:32:32.955543+0000","last_clean":"2024-09-17T23:32:32.955543+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:32.955543+0000","last_undegraded":"2024-09-17T23:32:32.955543+0000","last_fullsized":"2024-09-17T23:32:32.955543+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:23:15.468503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"174'953","reported_seq":11755,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.954268+0000","last_change":"2024-09-17T23:32:17.873346+0000","last_active":"2024-09-17T23:32:32.954268+0000","last_peered":"2024-09-17T23:32:32.954268+0000","last_clean":"2024-09-17T23:32:32.954268+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:32.954268+0000","last_undegraded":"2024-09-17T23:32:32.954268+0000","last_fullsized":"2024-09-17T23:32:32.954268+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:06:59.962287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"172'510","reported_seq":5624,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955422+0000","last_change":"2024-09-17T23:32:17.873091+0000","last_active":"2024-09-17T23:32:32.955422+0000","last_peered":"2024-09-17T23:32:32.955422+0000","last_clean":"2024-09-17T23:32:32.955422+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:32.955422+0000","last_undegraded":"2024-09-17T23:32:32.955422+0000","last_fullsized":"2024-09-17T23:32:32.955422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:28:21.353619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"171'436","reported_seq":9861,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959619+0000","last_change":"2024-09-17T23:32:22.878491+0000","last_active":"2024-09-17T23:32:32.959619+0000","last_peered":"2024-09-17T23:32:32.959619+0000","last_clean":"2024-09-17T23:32:32.959619+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:32.959619+0000","last_undegraded":"2024-09-17T23:32:32.959619+0000","last_fullsized":"2024-09-17T23:32:32.959619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:52.466160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"170'749","reported_seq":10350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959730+0000","last_change":"2024-09-17T23:32:26.876897+0000","last_active":"2024-09-17T23:32:32.959730+0000","last_peered":"2024-09-17T23:32:32.959730+0000","last_clean":"2024-09-17T23:32:32.959730+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:32.959730+0000","last_undegraded":"2024-09-17T23:32:32.959730+0000","last_fullsized":"2024-09-17T23:32:32.959730+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:25:42.259419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"173'386","reported_seq":5130,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955226+0000","last_change":"2024-09-17T23:32:26.876451+0000","last_active":"2024-09-17T23:32:32.955226+0000","last_peered":"2024-09-17T23:32:32.955226+0000","last_clean":"2024-09-17T23:32:32.955226+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:32.955226+0000","last_undegraded":"2024-09-17T23:32:32.955226+0000","last_fullsized":"2024-09-17T23:32:32.955226+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:28.427580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"171'917","reported_seq":7998,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959251+0000","last_change":"2024-09-17T23:32:23.878627+0000","last_active":"2024-09-17T23:32:32.959251+0000","last_peered":"2024-09-17T23:32:32.959251+0000","last_clean":"2024-09-17T23:32:32.959251+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:32.959251+0000","last_undegraded":"2024-09-17T23:32:32.959251+0000","last_fullsized":"2024-09-17T23:32:32.959251+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:59:13.949860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"173'380","reported_seq":8679,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.955044+0000","last_change":"2024-09-17T23:32:17.871775+0000","last_active":"2024-09-17T23:32:32.955044+0000","last_peered":"2024-09-17T23:32:32.955044+0000","last_clean":"2024-09-17T23:32:32.955044+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:32.955044+0000","last_undegraded":"2024-09-17T23:32:32.955044+0000","last_fullsized":"2024-09-17T23:32:32.955044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:41.522918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"172'447","reported_seq":5925,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959067+0000","last_change":"2024-09-17T23:32:26.874708+0000","last_active":"2024-09-17T23:32:32.959067+0000","last_peered":"2024-09-17T23:32:32.959067+0000","last_clean":"2024-09-17T23:32:32.959067+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:32.959067+0000","last_undegraded":"2024-09-17T23:32:32.959067+0000","last_fullsized":"2024-09-17T23:32:32.959067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:23.493952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"174'605","reported_seq":15334,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:32.959350+0000","last_change":"2024-09-17T23:32:24.890401+0000","last_active":"2024-09-17T23:32:32.959350+0000","last_peered":"2024-09-17T23:32:32.959350+0000","last_clean":"2024-09-17T23:32:32.959350+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:32.959350+0000","last_undegraded":"2024-09-17T23:32:32.959350+0000","last_fullsized":"2024-09-17T23:32:32.959350+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:29:14.582629+0000","last_clean_scrub_stamp":"2024-09-17T23:29:14.582629+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:28:18.258055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19120,"ondisk_log_size":19120,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500692,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43140,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":40812,"kb_avail":94328700,"statfs":{"total":96636764160,"available":96592588800,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":41792171},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640304,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51468,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320372,"statfs":{"total":96636764160,"available":96584060928,"internally_reserved":0,"allocated":2277376,"data_stored":963969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640304,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77832,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294008,"statfs":{"total":96636764160,"available":96557064192,"internally_reserved":0,"allocated":2273280,"data_stored":963950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640304,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37576,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334264,"statfs":{"total":96636764160,"available":96598286336,"internally_reserved":0,"allocated":2273280,"data_stored":963950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640305,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2318336,"data_stored":1554337,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640305,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69508,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302332,"statfs":{"total":96636764160,"available":96565587968,"internally_reserved":0,"allocated":2277376,"data_stored":963969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:32:42.412 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.412 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.412 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.412 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.412 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.413 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.414 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.415 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T23:29:04.482750+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=4, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T23:28:58.457979+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=28, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.416 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T23:29:14.582629+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=29, tm_sec=14, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=23, tm_min=32, tm_sec=38, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T23:32:42.417 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T23:33:02.418 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T23:33:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-17T23:33:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-17T23:33:02.735 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-17T23:33:02.405360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":268,"num_object_clones":0,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":268,"num_whiteouts":260,"num_read":314260,"num_read_kb":1327788,"num_write":20262,"num_write_kb":483384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8896,"num_bytes_recovered":262172026,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19506,"ondisk_log_size":19506,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":330968,"kb_used_data":13432,"kb_used_omap":27,"kb_used_meta":317412,"kb_avail":565900072,"statfs":{"total":579820584960,"available":579481673728,"internally_reserved":0,"allocated":13754368,"data_stored":6964512,"data_compressed":10720,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":27768,"internal_metadata":325030792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002775"},"pg_stats":[{"pgid":"3.1f","version":"189'469","reported_seq":6262,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:45.070996+0000","last_change":"2024-09-17T23:32:45.071026+0000","last_active":"2024-09-17T23:32:45.070996+0000","last_peered":"2024-09-17T23:32:45.070996+0000","last_clean":"2024-09-17T23:32:45.070996+0000","last_became_active":"2024-09-17T23:32:32.961283+0000","last_became_peered":"2024-09-17T23:32:32.961283+0000","last_unstale":"2024-09-17T23:32:45.070996+0000","last_undegraded":"2024-09-17T23:32:45.070996+0000","last_fullsized":"2024-09-17T23:32:45.070996+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'469","last_scrub_stamp":"2024-09-17T23:32:45.070912+0000","last_deep_scrub":"189'469","last_deep_scrub_stamp":"2024-09-17T23:32:45.070912+0000","last_clean_scrub_stamp":"2024-09-17T23:32:45.070912+0000","objects_scrubbed":4,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:12.098714+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0023360019999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":7636,"num_read_kb":66625,"num_write":494,"num_write_kb":20596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":23820074,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'586","reported_seq":9368,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:45.906619+0000","last_change":"2024-09-17T23:32:45.906700+0000","last_active":"2024-09-17T23:32:45.906619+0000","last_peered":"2024-09-17T23:32:45.906619+0000","last_clean":"2024-09-17T23:32:45.906619+0000","last_became_active":"2024-09-17T23:32:22.876138+0000","last_became_peered":"2024-09-17T23:32:22.876138+0000","last_unstale":"2024-09-17T23:32:45.906619+0000","last_undegraded":"2024-09-17T23:32:45.906619+0000","last_fullsized":"2024-09-17T23:32:45.906619+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'586","last_scrub_stamp":"2024-09-17T23:32:45.906521+0000","last_deep_scrub":"189'586","last_deep_scrub_stamp":"2024-09-17T23:32:45.906521+0000","last_clean_scrub_stamp":"2024-09-17T23:32:45.906521+0000","objects_scrubbed":8,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:20.274252+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0048347980000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":9387,"num_read_kb":55133,"num_write":575,"num_write_kb":19050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":29281131,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"189'382","reported_seq":6003,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:44.876371+0000","last_change":"2024-09-17T23:32:44.876417+0000","last_active":"2024-09-17T23:32:44.876371+0000","last_peered":"2024-09-17T23:32:44.876371+0000","last_clean":"2024-09-17T23:32:44.876371+0000","last_became_active":"2024-09-17T23:32:26.873095+0000","last_became_peered":"2024-09-17T23:32:26.873095+0000","last_unstale":"2024-09-17T23:32:44.876371+0000","last_undegraded":"2024-09-17T23:32:44.876371+0000","last_fullsized":"2024-09-17T23:32:44.876371+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'382","last_scrub_stamp":"2024-09-17T23:32:44.876264+0000","last_deep_scrub":"189'382","last_deep_scrub_stamp":"2024-09-17T23:32:44.876264+0000","last_clean_scrub_stamp":"2024-09-17T23:32:44.876264+0000","objects_scrubbed":9,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:20:33.156355+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0031074179999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4805,"num_read_kb":41204,"num_write":330,"num_write_kb":14202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":13005979,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"189'476","reported_seq":9416,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:46.553175+0000","last_change":"2024-09-17T23:32:46.553211+0000","last_active":"2024-09-17T23:32:46.553175+0000","last_peered":"2024-09-17T23:32:46.553175+0000","last_clean":"2024-09-17T23:32:46.553175+0000","last_became_active":"2024-09-17T23:32:24.888332+0000","last_became_peered":"2024-09-17T23:32:24.888332+0000","last_unstale":"2024-09-17T23:32:46.553175+0000","last_undegraded":"2024-09-17T23:32:46.553175+0000","last_fullsized":"2024-09-17T23:32:46.553175+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'476","last_scrub_stamp":"2024-09-17T23:32:46.553085+0000","last_deep_scrub":"189'476","last_deep_scrub_stamp":"2024-09-17T23:32:46.553085+0000","last_clean_scrub_stamp":"2024-09-17T23:32:46.553085+0000","objects_scrubbed":6,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:49.986387+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0047482130000000003,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":8565,"num_read_kb":40311,"num_write":567,"num_write_kb":12352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":16272110,"num_keys_recovered":392,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"189'468","reported_seq":19766,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:43.899601+0000","last_change":"2024-09-17T23:32:43.899664+0000","last_active":"2024-09-17T23:32:43.899601+0000","last_peered":"2024-09-17T23:32:43.899601+0000","last_clean":"2024-09-17T23:32:43.899601+0000","last_became_active":"2024-09-17T23:31:56.864807+0000","last_became_peered":"2024-09-17T23:31:56.864807+0000","last_unstale":"2024-09-17T23:32:43.899601+0000","last_undegraded":"2024-09-17T23:32:43.899601+0000","last_fullsized":"2024-09-17T23:32:43.899601+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'468","last_scrub_stamp":"2024-09-17T23:32:43.899509+0000","last_deep_scrub":"189'468","last_deep_scrub_stamp":"2024-09-17T23:32:43.899509+0000","last_clean_scrub_stamp":"2024-09-17T23:32:43.899509+0000","objects_scrubbed":16,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:39:38.393693+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0025163059999999998,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":18896,"num_read_kb":58434,"num_write":479,"num_write_kb":23023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4606028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"189'890","reported_seq":7779,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:42.605730+0000","last_change":"2024-09-17T23:32:42.605766+0000","last_active":"2024-09-17T23:32:42.605730+0000","last_peered":"2024-09-17T23:32:42.605730+0000","last_clean":"2024-09-17T23:32:42.605730+0000","last_became_active":"2024-09-17T23:32:22.879272+0000","last_became_peered":"2024-09-17T23:32:22.879272+0000","last_unstale":"2024-09-17T23:32:42.605730+0000","last_undegraded":"2024-09-17T23:32:42.605730+0000","last_fullsized":"2024-09-17T23:32:42.605730+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'890","last_scrub_stamp":"2024-09-17T23:32:42.605663+0000","last_deep_scrub":"189'890","last_deep_scrub_stamp":"2024-09-17T23:32:42.605663+0000","last_clean_scrub_stamp":"2024-09-17T23:32:42.605663+0000","objects_scrubbed":8,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:00.489132+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002570115,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":12095,"num_read_kb":30992,"num_write":1211,"num_write_kb":6066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":7389482,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"189'847","reported_seq":6818,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:46.074513+0000","last_change":"2024-09-17T23:32:46.074553+0000","last_active":"2024-09-17T23:32:46.074513+0000","last_peered":"2024-09-17T23:32:46.074513+0000","last_clean":"2024-09-17T23:32:46.074513+0000","last_became_active":"2024-09-17T23:31:57.866355+0000","last_became_peered":"2024-09-17T23:31:57.866355+0000","last_unstale":"2024-09-17T23:32:46.074513+0000","last_undegraded":"2024-09-17T23:32:46.074513+0000","last_fullsized":"2024-09-17T23:32:46.074513+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'847","last_scrub_stamp":"2024-09-17T23:32:46.074422+0000","last_deep_scrub":"189'847","last_deep_scrub_stamp":"2024-09-17T23:32:46.074422+0000","last_clean_scrub_stamp":"2024-09-17T23:32:46.074422+0000","objects_scrubbed":3,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:50:29.441413+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0047661229999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8711,"num_read_kb":17692,"num_write":935,"num_write_kb":3502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":497,"num_bytes_recovered":4937592,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'497","reported_seq":4585,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:41.871125+0000","last_change":"2024-09-17T23:32:41.871174+0000","last_active":"2024-09-17T23:32:41.871125+0000","last_peered":"2024-09-17T23:32:41.871125+0000","last_clean":"2024-09-17T23:32:41.871125+0000","last_became_active":"2024-09-17T23:29:15.600338+0000","last_became_peered":"2024-09-17T23:29:15.600338+0000","last_unstale":"2024-09-17T23:32:41.871125+0000","last_undegraded":"2024-09-17T23:32:41.871125+0000","last_fullsized":"2024-09-17T23:32:41.871125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'497","last_scrub_stamp":"2024-09-17T23:32:41.871027+0000","last_deep_scrub":"189'497","last_deep_scrub_stamp":"2024-09-17T23:32:41.871027+0000","last_clean_scrub_stamp":"2024-09-17T23:32:41.871027+0000","objects_scrubbed":10,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:09:10.962065+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001562662,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":5193,"num_read_kb":22367,"num_write":468,"num_write_kb":20039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"189'679","reported_seq":8347,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:43.103745+0000","last_change":"2024-09-17T23:32:43.103804+0000","last_active":"2024-09-17T23:32:43.103745+0000","last_peered":"2024-09-17T23:32:43.103745+0000","last_clean":"2024-09-17T23:32:43.103745+0000","last_became_active":"2024-09-17T23:31:12.055949+0000","last_became_peered":"2024-09-17T23:31:12.055949+0000","last_unstale":"2024-09-17T23:32:43.103745+0000","last_undegraded":"2024-09-17T23:32:43.103745+0000","last_fullsized":"2024-09-17T23:32:43.103745+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'679","last_scrub_stamp":"2024-09-17T23:32:43.103636+0000","last_deep_scrub":"189'679","last_deep_scrub_stamp":"2024-09-17T23:32:43.103636+0000","last_clean_scrub_stamp":"2024-09-17T23:32:43.103636+0000","objects_scrubbed":4,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:56:10.559736+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0017433290000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":11107,"num_read_kb":29109,"num_write":574,"num_write_kb":13280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":6740830,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'400","reported_seq":4588,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:41.871303+0000","last_change":"2024-09-17T23:32:41.871335+0000","last_active":"2024-09-17T23:32:41.871303+0000","last_peered":"2024-09-17T23:32:41.871303+0000","last_clean":"2024-09-17T23:32:41.871303+0000","last_became_active":"2024-09-17T23:32:26.870415+0000","last_became_peered":"2024-09-17T23:32:26.870415+0000","last_unstale":"2024-09-17T23:32:41.871303+0000","last_undegraded":"2024-09-17T23:32:41.871303+0000","last_fullsized":"2024-09-17T23:32:41.871303+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'400","last_scrub_stamp":"2024-09-17T23:32:41.871225+0000","last_deep_scrub":"189'400","last_deep_scrub_stamp":"2024-09-17T23:32:41.871225+0000","last_clean_scrub_stamp":"2024-09-17T23:32:41.871225+0000","objects_scrubbed":6,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:07.005175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0040051720000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":4601,"num_read_kb":31330,"num_write":384,"num_write_kb":10184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":8161984,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'640","reported_seq":13077,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:40.444613+0000","last_change":"2024-09-17T23:32:40.444646+0000","last_active":"2024-09-17T23:32:40.444613+0000","last_peered":"2024-09-17T23:32:40.444613+0000","last_clean":"2024-09-17T23:32:40.444613+0000","last_became_active":"2024-09-17T23:29:15.600995+0000","last_became_peered":"2024-09-17T23:29:15.600995+0000","last_unstale":"2024-09-17T23:32:40.444613+0000","last_undegraded":"2024-09-17T23:32:40.444613+0000","last_fullsized":"2024-09-17T23:32:40.444613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'640","last_scrub_stamp":"2024-09-17T23:32:40.444535+0000","last_deep_scrub":"189'640","last_deep_scrub_stamp":"2024-09-17T23:32:40.444535+0000","last_clean_scrub_stamp":"2024-09-17T23:32:40.444535+0000","objects_scrubbed":5,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:09:42.322530+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0045893289999999996,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":14723,"num_read_kb":31747,"num_write":655,"num_write_kb":6556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"189'637","reported_seq":8380,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:49.616111+0000","last_change":"2024-09-17T23:32:49.616150+0000","last_active":"2024-09-17T23:32:49.616111+0000","last_peered":"2024-09-17T23:32:49.616111+0000","last_clean":"2024-09-17T23:32:49.616111+0000","last_became_active":"2024-09-17T23:32:26.871541+0000","last_became_peered":"2024-09-17T23:32:26.871541+0000","last_unstale":"2024-09-17T23:32:49.616111+0000","last_undegraded":"2024-09-17T23:32:49.616111+0000","last_fullsized":"2024-09-17T23:32:49.616111+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'637","last_scrub_stamp":"2024-09-17T23:32:49.616020+0000","last_deep_scrub":"189'637","last_deep_scrub_stamp":"2024-09-17T23:32:49.616020+0000","last_clean_scrub_stamp":"2024-09-17T23:32:49.616020+0000","objects_scrubbed":7,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:09.046485+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0053089779999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8986,"num_read_kb":34057,"num_write":520,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":2314946,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:48.574841+0000","last_change":"2024-09-17T23:32:48.574866+0000","last_active":"2024-09-17T23:32:48.574841+0000","last_peered":"2024-09-17T23:32:48.574841+0000","last_clean":"2024-09-17T23:32:48.574841+0000","last_became_active":"2024-09-17T23:32:24.890381+0000","last_became_peered":"2024-09-17T23:32:24.890381+0000","last_unstale":"2024-09-17T23:32:48.574841+0000","last_undegraded":"2024-09-17T23:32:48.574841+0000","last_fullsized":"2024-09-17T23:32:48.574841+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:32:48.574749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:32:48.574749+0000","last_clean_scrub_stamp":"2024-09-17T23:32:48.574749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:50.513697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"189'671","reported_seq":5286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:43.127319+0000","last_change":"2024-09-17T23:32:43.127387+0000","last_active":"2024-09-17T23:32:43.127319+0000","last_peered":"2024-09-17T23:32:43.127319+0000","last_clean":"2024-09-17T23:32:43.127319+0000","last_became_active":"2024-09-17T23:29:15.601643+0000","last_became_peered":"2024-09-17T23:29:15.601643+0000","last_unstale":"2024-09-17T23:32:43.127319+0000","last_undegraded":"2024-09-17T23:32:43.127319+0000","last_fullsized":"2024-09-17T23:32:43.127319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'671","last_scrub_stamp":"2024-09-17T23:32:43.127217+0000","last_deep_scrub":"189'671","last_deep_scrub_stamp":"2024-09-17T23:32:43.127217+0000","last_clean_scrub_stamp":"2024-09-17T23:32:43.127217+0000","objects_scrubbed":7,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:39:37.431319+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0033074319999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":6036,"num_read_kb":22666,"num_write":904,"num_write_kb":9111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:42.077151+0000","last_change":"2024-09-17T23:32:42.077198+0000","last_active":"2024-09-17T23:32:42.077151+0000","last_peered":"2024-09-17T23:32:42.077151+0000","last_clean":"2024-09-17T23:32:42.077151+0000","last_became_active":"2024-09-17T23:29:05.498531+0000","last_became_peered":"2024-09-17T23:29:05.498531+0000","last_unstale":"2024-09-17T23:32:42.077151+0000","last_undegraded":"2024-09-17T23:32:42.077151+0000","last_fullsized":"2024-09-17T23:32:42.077151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:32:42.077054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:32:42.077054+0000","last_clean_scrub_stamp":"2024-09-17T23:32:42.077054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:25:09.194274+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00039068700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'460","reported_seq":12769,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:44.084354+0000","last_change":"2024-09-17T23:32:44.084427+0000","last_active":"2024-09-17T23:32:44.084354+0000","last_peered":"2024-09-17T23:32:44.084354+0000","last_clean":"2024-09-17T23:32:44.084354+0000","last_became_active":"2024-09-17T23:29:15.602574+0000","last_became_peered":"2024-09-17T23:29:15.602574+0000","last_unstale":"2024-09-17T23:32:44.084354+0000","last_undegraded":"2024-09-17T23:32:44.084354+0000","last_fullsized":"2024-09-17T23:32:44.084354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'460","last_scrub_stamp":"2024-09-17T23:32:44.084237+0000","last_deep_scrub":"189'460","last_deep_scrub_stamp":"2024-09-17T23:32:44.084237+0000","last_clean_scrub_stamp":"2024-09-17T23:32:44.084237+0000","objects_scrubbed":7,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:20:31.415204+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.003791886,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":13228,"num_read_kb":60651,"num_write":441,"num_write_kb":30575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":330,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:44.539939+0000","last_change":"2024-09-17T23:32:44.539966+0000","last_active":"2024-09-17T23:32:44.539939+0000","last_peered":"2024-09-17T23:32:44.539939+0000","last_clean":"2024-09-17T23:32:44.539939+0000","last_became_active":"2024-09-17T23:32:24.883003+0000","last_became_peered":"2024-09-17T23:32:24.883003+0000","last_unstale":"2024-09-17T23:32:44.539939+0000","last_undegraded":"2024-09-17T23:32:44.539939+0000","last_fullsized":"2024-09-17T23:32:44.539939+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:32:44.539827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:32:44.539827+0000","last_clean_scrub_stamp":"2024-09-17T23:32:44.539827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:18:54.712082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"189'432","reported_seq":7452,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:45.053221+0000","last_change":"2024-09-17T23:32:45.053272+0000","last_active":"2024-09-17T23:32:45.053221+0000","last_peered":"2024-09-17T23:32:45.053221+0000","last_clean":"2024-09-17T23:32:45.053221+0000","last_became_active":"2024-09-17T23:31:37.866203+0000","last_became_peered":"2024-09-17T23:31:37.866203+0000","last_unstale":"2024-09-17T23:32:45.053221+0000","last_undegraded":"2024-09-17T23:32:45.053221+0000","last_fullsized":"2024-09-17T23:32:45.053221+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'432","last_scrub_stamp":"2024-09-17T23:32:45.053117+0000","last_deep_scrub":"189'432","last_deep_scrub_stamp":"2024-09-17T23:32:45.053117+0000","last_clean_scrub_stamp":"2024-09-17T23:32:45.053117+0000","objects_scrubbed":16,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:01.991654+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0039376599999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4612,"num_read_kb":19723,"num_write":450,"num_write_kb":6251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":10107019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":191,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:44.112114+0000","last_change":"2024-09-17T23:32:44.112140+0000","last_active":"2024-09-17T23:32:44.112114+0000","last_peered":"2024-09-17T23:32:44.112114+0000","last_clean":"2024-09-17T23:32:44.112114+0000","last_became_active":"2024-09-17T23:32:32.958379+0000","last_became_peered":"2024-09-17T23:32:32.958379+0000","last_unstale":"2024-09-17T23:32:44.112114+0000","last_undegraded":"2024-09-17T23:32:44.112114+0000","last_fullsized":"2024-09-17T23:32:44.112114+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:32:44.112021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:32:44.112021+0000","last_clean_scrub_stamp":"2024-09-17T23:32:44.112021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:45.422891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042311099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'411","reported_seq":10687,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:42.895382+0000","last_change":"2024-09-17T23:32:42.895422+0000","last_active":"2024-09-17T23:32:42.895382+0000","last_peered":"2024-09-17T23:32:42.895382+0000","last_clean":"2024-09-17T23:32:42.895382+0000","last_became_active":"2024-09-17T23:32:26.870347+0000","last_became_peered":"2024-09-17T23:32:26.870347+0000","last_unstale":"2024-09-17T23:32:42.895382+0000","last_undegraded":"2024-09-17T23:32:42.895382+0000","last_fullsized":"2024-09-17T23:32:42.895382+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'411","last_scrub_stamp":"2024-09-17T23:32:42.895296+0000","last_deep_scrub":"189'411","last_deep_scrub_stamp":"2024-09-17T23:32:42.895296+0000","last_clean_scrub_stamp":"2024-09-17T23:32:42.895296+0000","objects_scrubbed":10,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:45:40.514863+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.003955493,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10202,"num_read_kb":27184,"num_write":439,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":2836058,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"189'706","reported_seq":8012,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:45.507098+0000","last_change":"2024-09-17T23:32:45.507197+0000","last_active":"2024-09-17T23:32:45.507098+0000","last_peered":"2024-09-17T23:32:45.507098+0000","last_clean":"2024-09-17T23:32:45.507098+0000","last_became_active":"2024-09-17T23:32:24.882634+0000","last_became_peered":"2024-09-17T23:32:24.882634+0000","last_unstale":"2024-09-17T23:32:45.507098+0000","last_undegraded":"2024-09-17T23:32:45.507098+0000","last_fullsized":"2024-09-17T23:32:45.507098+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'706","last_scrub_stamp":"2024-09-17T23:32:45.507006+0000","last_deep_scrub":"189'706","last_deep_scrub_stamp":"2024-09-17T23:32:45.507006+0000","last_clean_scrub_stamp":"2024-09-17T23:32:45.507006+0000","objects_scrubbed":17,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:14:52.593234+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0040670339999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7627,"num_read_kb":30055,"num_write":690,"num_write_kb":9219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2512896,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"189'3","reported_seq":426,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:42.428549+0000","last_change":"2024-09-17T23:32:42.428582+0000","last_active":"2024-09-17T23:32:42.428549+0000","last_peered":"2024-09-17T23:32:42.428549+0000","last_clean":"2024-09-17T23:32:42.428549+0000","last_became_active":"2024-09-17T23:31:55.008812+0000","last_became_peered":"2024-09-17T23:31:55.008812+0000","last_unstale":"2024-09-17T23:32:42.428549+0000","last_undegraded":"2024-09-17T23:32:42.428549+0000","last_fullsized":"2024-09-17T23:32:42.428549+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'3","last_scrub_stamp":"2024-09-17T23:32:42.428472+0000","last_deep_scrub":"189'3","last_deep_scrub_stamp":"2024-09-17T23:32:42.428472+0000","last_clean_scrub_stamp":"2024-09-17T23:32:42.428472+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:03.441101+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00057254199999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"189'710","reported_seq":12545,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:46.066449+0000","last_change":"2024-09-17T23:32:46.066516+0000","last_active":"2024-09-17T23:32:46.066449+0000","last_peered":"2024-09-17T23:32:46.066449+0000","last_clean":"2024-09-17T23:32:46.066449+0000","last_became_active":"2024-09-17T23:32:32.961961+0000","last_became_peered":"2024-09-17T23:32:32.961961+0000","last_unstale":"2024-09-17T23:32:46.066449+0000","last_undegraded":"2024-09-17T23:32:46.066449+0000","last_fullsized":"2024-09-17T23:32:46.066449+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'710","last_scrub_stamp":"2024-09-17T23:32:46.066355+0000","last_deep_scrub":"189'710","last_deep_scrub_stamp":"2024-09-17T23:32:46.066355+0000","last_clean_scrub_stamp":"2024-09-17T23:32:46.066355+0000","objects_scrubbed":11,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:29.194362+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0020023369999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13034,"num_read_kb":56604,"num_write":744,"num_write_kb":10229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":662,"num_bytes_recovered":11604246,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":376,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:42.854493+0000","last_change":"2024-09-17T23:32:42.854540+0000","last_active":"2024-09-17T23:32:42.854493+0000","last_peered":"2024-09-17T23:32:42.854493+0000","last_clean":"2024-09-17T23:32:42.854493+0000","last_became_active":"2024-09-17T23:31:12.051903+0000","last_became_peered":"2024-09-17T23:31:12.051903+0000","last_unstale":"2024-09-17T23:32:42.854493+0000","last_undegraded":"2024-09-17T23:32:42.854493+0000","last_fullsized":"2024-09-17T23:32:42.854493+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:32:42.854377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:32:42.854377+0000","last_clean_scrub_stamp":"2024-09-17T23:32:42.854377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:18.356987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036477800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"189'960","reported_seq":9738,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:48.127384+0000","last_change":"2024-09-17T23:32:48.127475+0000","last_active":"2024-09-17T23:32:48.127384+0000","last_peered":"2024-09-17T23:32:48.127384+0000","last_clean":"2024-09-17T23:32:48.127384+0000","last_became_active":"2024-09-17T23:32:23.878547+0000","last_became_peered":"2024-09-17T23:32:23.878547+0000","last_unstale":"2024-09-17T23:32:48.127384+0000","last_undegraded":"2024-09-17T23:32:48.127384+0000","last_fullsized":"2024-09-17T23:32:48.127384+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'960","last_scrub_stamp":"2024-09-17T23:32:48.127277+0000","last_deep_scrub":"189'960","last_deep_scrub_stamp":"2024-09-17T23:32:48.127277+0000","last_clean_scrub_stamp":"2024-09-17T23:32:48.127277+0000","objects_scrubbed":11,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:47.792940+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0040252539999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11953,"num_read_kb":16470,"num_write":742,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":5006572,"num_keys_recovered":183,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:40.833616+0000","last_change":"2024-09-17T23:32:40.833635+0000","last_active":"2024-09-17T23:32:40.833616+0000","last_peered":"2024-09-17T23:32:40.833616+0000","last_clean":"2024-09-17T23:32:40.833616+0000","last_became_active":"2024-09-17T23:32:20.870808+0000","last_became_peered":"2024-09-17T23:32:20.870808+0000","last_unstale":"2024-09-17T23:32:40.833616+0000","last_undegraded":"2024-09-17T23:32:40.833616+0000","last_fullsized":"2024-09-17T23:32:40.833616+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T23:32:40.833526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T23:32:40.833526+0000","last_clean_scrub_stamp":"2024-09-17T23:32:40.833526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:28.037105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"189'730","reported_seq":11681,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:46.869362+0000","last_change":"2024-09-17T23:32:46.869403+0000","last_active":"2024-09-17T23:32:46.869362+0000","last_peered":"2024-09-17T23:32:46.869362+0000","last_clean":"2024-09-17T23:32:46.869362+0000","last_became_active":"2024-09-17T23:32:24.880530+0000","last_became_peered":"2024-09-17T23:32:24.880530+0000","last_unstale":"2024-09-17T23:32:46.869362+0000","last_undegraded":"2024-09-17T23:32:46.869362+0000","last_fullsized":"2024-09-17T23:32:46.869362+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'730","last_scrub_stamp":"2024-09-17T23:32:46.869274+0000","last_deep_scrub":"189'730","last_deep_scrub_stamp":"2024-09-17T23:32:46.869274+0000","last_clean_scrub_stamp":"2024-09-17T23:32:46.869274+0000","objects_scrubbed":13,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:25.816478+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001601522,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":11237,"num_read_kb":79320,"num_write":788,"num_write_kb":59945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":18861540,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"189'2","reported_seq":38,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:45.590996+0000","last_change":"2024-09-17T23:32:45.591026+0000","last_active":"2024-09-17T23:32:45.590996+0000","last_peered":"2024-09-17T23:32:45.590996+0000","last_clean":"2024-09-17T23:32:45.590996+0000","last_became_active":"2024-09-17T23:32:24.886687+0000","last_became_peered":"2024-09-17T23:32:24.886687+0000","last_unstale":"2024-09-17T23:32:45.590996+0000","last_undegraded":"2024-09-17T23:32:45.590996+0000","last_fullsized":"2024-09-17T23:32:45.590996+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'2","last_scrub_stamp":"2024-09-17T23:32:45.590911+0000","last_deep_scrub":"189'2","last_deep_scrub_stamp":"2024-09-17T23:32:45.590911+0000","last_clean_scrub_stamp":"2024-09-17T23:32:45.590911+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:15:46.679799+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010275550000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"189'119","reported_seq":465,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:41.574763+0000","last_change":"2024-09-17T23:32:41.574790+0000","last_active":"2024-09-17T23:32:41.574763+0000","last_peered":"2024-09-17T23:32:41.574763+0000","last_clean":"2024-09-17T23:32:41.574763+0000","last_became_active":"2024-09-17T23:32:22.878677+0000","last_became_peered":"2024-09-17T23:32:22.878677+0000","last_unstale":"2024-09-17T23:32:41.574763+0000","last_undegraded":"2024-09-17T23:32:41.574763+0000","last_fullsized":"2024-09-17T23:32:41.574763+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'119","last_scrub_stamp":"2024-09-17T23:32:41.574687+0000","last_deep_scrub":"189'119","last_deep_scrub_stamp":"2024-09-17T23:32:41.574687+0000","last_clean_scrub_stamp":"2024-09-17T23:32:41.574687+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:16:34.543772+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"189'501","reported_seq":15247,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:47.104506+0000","last_change":"2024-09-17T23:32:47.104547+0000","last_active":"2024-09-17T23:32:47.104506+0000","last_peered":"2024-09-17T23:32:47.104506+0000","last_clean":"2024-09-17T23:32:47.104506+0000","last_became_active":"2024-09-17T23:32:23.876672+0000","last_became_peered":"2024-09-17T23:32:23.876672+0000","last_unstale":"2024-09-17T23:32:47.104506+0000","last_undegraded":"2024-09-17T23:32:47.104506+0000","last_fullsized":"2024-09-17T23:32:47.104506+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'501","last_scrub_stamp":"2024-09-17T23:32:47.104415+0000","last_deep_scrub":"189'501","last_deep_scrub_stamp":"2024-09-17T23:32:47.104415+0000","last_clean_scrub_stamp":"2024-09-17T23:32:47.104415+0000","objects_scrubbed":10,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:21:25.501136+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0032261820000000002,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":15467,"num_read_kb":74193,"num_write":427,"num_write_kb":7875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":2405645,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'748","reported_seq":11766,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:43.458171+0000","last_change":"2024-09-17T23:32:43.458212+0000","last_active":"2024-09-17T23:32:43.458171+0000","last_peered":"2024-09-17T23:32:43.458171+0000","last_clean":"2024-09-17T23:32:43.458171+0000","last_became_active":"2024-09-17T23:31:55.867195+0000","last_became_peered":"2024-09-17T23:31:55.867195+0000","last_unstale":"2024-09-17T23:32:43.458171+0000","last_undegraded":"2024-09-17T23:32:43.458171+0000","last_fullsized":"2024-09-17T23:32:43.458171+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'748","last_scrub_stamp":"2024-09-17T23:32:43.458083+0000","last_deep_scrub":"189'748","last_deep_scrub_stamp":"2024-09-17T23:32:43.458083+0000","last_clean_scrub_stamp":"2024-09-17T23:32:43.458083+0000","objects_scrubbed":7,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:53.882980+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0020861009999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":11343,"num_read_kb":39222,"num_write":838,"num_write_kb":25907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":3355281,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"189'641","reported_seq":10184,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:46.502169+0000","last_change":"2024-09-17T23:32:46.502213+0000","last_active":"2024-09-17T23:32:46.502169+0000","last_peered":"2024-09-17T23:32:46.502169+0000","last_clean":"2024-09-17T23:32:46.502169+0000","last_became_active":"2024-09-17T23:32:26.874392+0000","last_became_peered":"2024-09-17T23:32:26.874392+0000","last_unstale":"2024-09-17T23:32:46.502169+0000","last_undegraded":"2024-09-17T23:32:46.502169+0000","last_fullsized":"2024-09-17T23:32:46.502169+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'641","last_scrub_stamp":"2024-09-17T23:32:46.502080+0000","last_deep_scrub":"189'641","last_deep_scrub_stamp":"2024-09-17T23:32:46.502080+0000","last_clean_scrub_stamp":"2024-09-17T23:32:46.502080+0000","objects_scrubbed":11,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:51:26.441515+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0030552349999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":10968,"num_read_kb":53306,"num_write":613,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4243261,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"189'957","reported_seq":11768,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:44.932004+0000","last_change":"2024-09-17T23:32:44.932045+0000","last_active":"2024-09-17T23:32:44.932004+0000","last_peered":"2024-09-17T23:32:44.932004+0000","last_clean":"2024-09-17T23:32:44.932004+0000","last_became_active":"2024-09-17T23:31:57.869346+0000","last_became_peered":"2024-09-17T23:31:57.869346+0000","last_unstale":"2024-09-17T23:32:44.932004+0000","last_undegraded":"2024-09-17T23:32:44.932004+0000","last_fullsized":"2024-09-17T23:32:44.932004+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'957","last_scrub_stamp":"2024-09-17T23:32:44.931912+0000","last_deep_scrub":"189'957","last_deep_scrub_stamp":"2024-09-17T23:32:44.931912+0000","last_clean_scrub_stamp":"2024-09-17T23:32:44.931912+0000","objects_scrubbed":4,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:57:53.083590+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0049057179999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":15192,"num_read_kb":61626,"num_write":914,"num_write_kb":11842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":4051046,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"189'518","reported_seq":5641,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:41.409453+0000","last_change":"2024-09-17T23:32:41.409512+0000","last_active":"2024-09-17T23:32:41.409453+0000","last_peered":"2024-09-17T23:32:41.409453+0000","last_clean":"2024-09-17T23:32:41.409453+0000","last_became_active":"2024-09-17T23:29:15.601970+0000","last_became_peered":"2024-09-17T23:29:15.601970+0000","last_unstale":"2024-09-17T23:32:41.409453+0000","last_undegraded":"2024-09-17T23:32:41.409453+0000","last_fullsized":"2024-09-17T23:32:41.409453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'518","last_scrub_stamp":"2024-09-17T23:32:41.409337+0000","last_deep_scrub":"189'518","last_deep_scrub_stamp":"2024-09-17T23:32:41.409337+0000","last_clean_scrub_stamp":"2024-09-17T23:32:41.409337+0000","objects_scrubbed":8,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:46.227279+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.005130089,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":6164,"num_read_kb":36966,"num_write":485,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"189'447","reported_seq":9881,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:40.546669+0000","last_change":"2024-09-17T23:32:40.546716+0000","last_active":"2024-09-17T23:32:40.546669+0000","last_peered":"2024-09-17T23:32:40.546669+0000","last_clean":"2024-09-17T23:32:40.546669+0000","last_became_active":"2024-09-17T23:32:22.876750+0000","last_became_peered":"2024-09-17T23:32:22.876750+0000","last_unstale":"2024-09-17T23:32:40.546669+0000","last_undegraded":"2024-09-17T23:32:40.546669+0000","last_fullsized":"2024-09-17T23:32:40.546669+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'447","last_scrub_stamp":"2024-09-17T23:32:40.546581+0000","last_deep_scrub":"189'447","last_deep_scrub_stamp":"2024-09-17T23:32:40.546581+0000","last_clean_scrub_stamp":"2024-09-17T23:32:40.546581+0000","objects_scrubbed":11,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:23.052935+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.003554078,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9089,"num_read_kb":42828,"num_write":472,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":15203077,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"189'756","reported_seq":10366,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:45.890211+0000","last_change":"2024-09-17T23:32:45.890280+0000","last_active":"2024-09-17T23:32:45.890211+0000","last_peered":"2024-09-17T23:32:45.890211+0000","last_clean":"2024-09-17T23:32:45.890211+0000","last_became_active":"2024-09-17T23:32:26.873717+0000","last_became_peered":"2024-09-17T23:32:26.873717+0000","last_unstale":"2024-09-17T23:32:45.890211+0000","last_undegraded":"2024-09-17T23:32:45.890211+0000","last_fullsized":"2024-09-17T23:32:45.890211+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'756","last_scrub_stamp":"2024-09-17T23:32:45.890107+0000","last_deep_scrub":"189'756","last_deep_scrub_stamp":"2024-09-17T23:32:45.890107+0000","last_clean_scrub_stamp":"2024-09-17T23:32:45.890107+0000","objects_scrubbed":7,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:13:45.162067+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0056914269999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10602,"num_read_kb":26592,"num_write":793,"num_write_kb":7201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":5513844,"num_keys_recovered":57,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'388","reported_seq":5141,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:47.515645+0000","last_change":"2024-09-17T23:32:47.515681+0000","last_active":"2024-09-17T23:32:47.515645+0000","last_peered":"2024-09-17T23:32:47.515645+0000","last_clean":"2024-09-17T23:32:47.515645+0000","last_became_active":"2024-09-17T23:32:26.874645+0000","last_became_peered":"2024-09-17T23:32:26.874645+0000","last_unstale":"2024-09-17T23:32:47.515645+0000","last_undegraded":"2024-09-17T23:32:47.515645+0000","last_fullsized":"2024-09-17T23:32:47.515645+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'388","last_scrub_stamp":"2024-09-17T23:32:47.515544+0000","last_deep_scrub":"189'388","last_deep_scrub_stamp":"2024-09-17T23:32:47.515544+0000","last_clean_scrub_stamp":"2024-09-17T23:32:47.515544+0000","objects_scrubbed":2,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:45.157111+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0026997779999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4851,"num_read_kb":27429,"num_write":393,"num_write_kb":5089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":5542111,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"189'920","reported_seq":8010,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:43.589166+0000","last_change":"2024-09-17T23:32:43.589203+0000","last_active":"2024-09-17T23:32:43.589166+0000","last_peered":"2024-09-17T23:32:43.589166+0000","last_clean":"2024-09-17T23:32:43.589166+0000","last_became_active":"2024-09-17T23:32:23.876741+0000","last_became_peered":"2024-09-17T23:32:23.876741+0000","last_unstale":"2024-09-17T23:32:43.589166+0000","last_undegraded":"2024-09-17T23:32:43.589166+0000","last_fullsized":"2024-09-17T23:32:43.589166+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'920","last_scrub_stamp":"2024-09-17T23:32:43.589063+0000","last_deep_scrub":"189'920","last_deep_scrub_stamp":"2024-09-17T23:32:43.589063+0000","last_clean_scrub_stamp":"2024-09-17T23:32:43.589063+0000","objects_scrubbed":3,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:15:01.277424+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0021124949999999998,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":9644,"num_read_kb":34087,"num_write":1220,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":8641256,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"189'385","reported_seq":8693,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:44.476047+0000","last_change":"2024-09-17T23:32:44.476087+0000","last_active":"2024-09-17T23:32:44.476047+0000","last_peered":"2024-09-17T23:32:44.476047+0000","last_clean":"2024-09-17T23:32:44.476047+0000","last_became_active":"2024-09-17T23:32:11.867862+0000","last_became_peered":"2024-09-17T23:32:11.867862+0000","last_unstale":"2024-09-17T23:32:44.476047+0000","last_undegraded":"2024-09-17T23:32:44.476047+0000","last_fullsized":"2024-09-17T23:32:44.476047+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'385","last_scrub_stamp":"2024-09-17T23:32:44.475957+0000","last_deep_scrub":"189'385","last_deep_scrub_stamp":"2024-09-17T23:32:44.475957+0000","last_clean_scrub_stamp":"2024-09-17T23:32:44.475957+0000","objects_scrubbed":5,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:02.578175+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0047097040000000003,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":8068,"num_read_kb":46907,"num_write":336,"num_write_kb":8554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":19670632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"189'458","reported_seq":5945,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:43.878853+0000","last_change":"2024-09-17T23:32:43.878909+0000","last_active":"2024-09-17T23:32:43.878853+0000","last_peered":"2024-09-17T23:32:43.878853+0000","last_clean":"2024-09-17T23:32:43.878853+0000","last_became_active":"2024-09-17T23:32:26.871613+0000","last_became_peered":"2024-09-17T23:32:26.871613+0000","last_unstale":"2024-09-17T23:32:43.878853+0000","last_undegraded":"2024-09-17T23:32:43.878853+0000","last_fullsized":"2024-09-17T23:32:43.878853+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'458","last_scrub_stamp":"2024-09-17T23:32:43.878751+0000","last_deep_scrub":"189'458","last_deep_scrub_stamp":"2024-09-17T23:32:43.878751+0000","last_clean_scrub_stamp":"2024-09-17T23:32:43.878751+0000","objects_scrubbed":11,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:03:19.880909+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0027140490000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5018,"num_read_kb":38008,"num_write":482,"num_write_kb":22143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8123392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"189'612","reported_seq":15350,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-17T23:32:47.584345+0000","last_change":"2024-09-17T23:32:47.584378+0000","last_active":"2024-09-17T23:32:47.584345+0000","last_peered":"2024-09-17T23:32:47.584345+0000","last_clean":"2024-09-17T23:32:47.584345+0000","last_became_active":"2024-09-17T23:32:24.888405+0000","last_became_peered":"2024-09-17T23:32:24.888405+0000","last_unstale":"2024-09-17T23:32:47.584345+0000","last_undegraded":"2024-09-17T23:32:47.584345+0000","last_fullsized":"2024-09-17T23:32:47.584345+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'612","last_scrub_stamp":"2024-09-17T23:32:47.584255+0000","last_deep_scrub":"189'612","last_deep_scrub_stamp":"2024-09-17T23:32:47.584255+0000","last_clean_scrub_stamp":"2024-09-17T23:32:47.584255+0000","objects_scrubbed":7,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:21:27.647487+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0035317399999999998,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":15104,"num_read_kb":74850,"num_write":738,"num_write_kb":33705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":16783163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":260,"num_read":314144,"num_read_kb":1327688,"num_write":20101,"num_write_kb":480966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8882,"num_bytes_recovered":260991195,"num_keys_recovered":1266,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0},"log_size":19382,"ondisk_log_size":19382,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":16498,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":107,"seq":459561500696,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2280,"kb_used_omap":19,"kb_used_meta":44972,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2334720,"data_stored":1554337,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":19797,"internal_metadata":46052011},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640308,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51456,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320384,"statfs":{"total":96636764160,"available":96584073216,"internally_reserved":0,"allocated":2277376,"data_stored":963969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640308,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77820,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294020,"statfs":{"total":96636764160,"available":96557076480,"internally_reserved":0,"allocated":2273280,"data_stored":963950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640308,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37564,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334276,"statfs":{"total":96636764160,"available":96598298624,"internally_reserved":0,"allocated":2273280,"data_stored":963950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640309,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2318336,"data_stored":1554337,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640309,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69504,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302336,"statfs":{"total":96636764160,"available":96565592064,"internally_reserved":0,"allocated":2277376,"data_stored":963969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":5778,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4942,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7598,"internal_metadata":0}]}} 2024-09-17T23:33:02.738 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T23:33:03.045 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T23:33:03.045 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T23:33:03.045 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T23:33:03.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.089 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T23:33:03.089 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T23:33:03.089 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.127 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T23:33:03.128 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T23:33:03.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.203 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T23:33:03.203 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T23:33:03.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.243 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T23:33:03.244 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T23:33:03.244 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.293 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T23:33:03.293 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T23:33:03.294 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.335 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T23:33:03.335 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T23:33:03.335 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T23:33:03.335 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.365 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T23:33:03.365 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T23:33:03.366 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T23:33:03.366 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.377 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T23:33:03.377 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T23:33:03.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.429 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T23:33:03.429 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T23:33:03.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T23:33:03.443 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T23:33:03.443 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T23:33:03.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T23:33:03.503 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:33:03.503 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T23:33:03.712 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:33:03.712 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T23:33:03.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:33:03.787 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T23:33:03.850 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:33:03.850 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T23:33:03.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:33:03.962 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T23:33:04.034 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:33:04.034 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T23:33:04.091 INFO:tasks.ceph:Archiving mon data... 2024-09-17T23:33:04.092 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/data/mon.a.tgz 2024-09-17T23:33:04.092 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-17T23:33:04.109 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.FFHDCj3PNG 2024-09-17T23:33:04.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FFHDCj3PNG 2024-09-17T23:33:04.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.FFHDCj3PNG 2024-09-17T23:33:04.343 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.FFHDCj3PNG is 361KB 2024-09-17T23:33:04.411 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.FFHDCj3PNG 2024-09-17T23:33:04.426 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/data/mon.c.tgz 2024-09-17T23:33:04.427 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-17T23:33:04.482 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.c3pq4cmxIs 2024-09-17T23:33:04.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.c3pq4cmxIs 2024-09-17T23:33:04.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.c3pq4cmxIs 2024-09-17T23:33:04.736 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.c3pq4cmxIs is 479KB 2024-09-17T23:33:04.833 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.c3pq4cmxIs 2024-09-17T23:33:04.849 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/data/mon.b.tgz 2024-09-17T23:33:04.849 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-17T23:33:04.869 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.ydfP3Tomf7 2024-09-17T23:33:04.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ydfP3Tomf7 2024-09-17T23:33:05.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.ydfP3Tomf7 2024-09-17T23:33:05.153 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.ydfP3Tomf7 is 479KB 2024-09-17T23:33:05.644 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.ydfP3Tomf7 2024-09-17T23:33:05.662 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T23:33:05.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T23:33:05.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T23:33:05.704 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T23:33:05.756 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T23:33:05.789 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T23:33:05.807 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T23:33:05.807 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T23:33:05.821 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi031/crash 2024-09-17T23:33:05.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T23:33:05.855 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi037/crash 2024-09-17T23:33:05.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T23:33:05.884 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi139/crash 2024-09-17T23:33:05.885 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T23:33:05.946 INFO:tasks.ceph:Compressing logs... 2024-09-17T23:33:05.946 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T23:33:05.949 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T23:33:05.951 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40644.log 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43284.log 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/objectstore_tool.40644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31716.log 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/objectstore_tool.43284.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41524.log 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.31716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31887.log 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31716.log.gz 2024-09-17T23:33:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/objectstore_tool.41524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31847.log 2024-09-17T23:33:05.994 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.31887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31887.log.gz 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42404.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.31847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31847.log.gz 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/objectstore_tool.42404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31807.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.31807.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31807.log.gz 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31760.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31927.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.31760.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31760.log.gz 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.31927.log: /var/log/ceph/ceph-osd.5.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31927.log.gz 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T23:33:05.995 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.40644.log.gz 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.43284.log.gz 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.3.log: 95.0% -- replaced with /var/log/ceph/objectstore_tool.41524.log.gz 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T23:33:05.996 INFO:teuthology.orchestra.run.smithi037.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.42404.log.gz 2024-09-17T23:33:06.070 INFO:teuthology.orchestra.run.smithi037.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T23:33:06.109 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29811.log 2024-09-17T23:33:06.109 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28095.log 2024-09-17T23:33:06.109 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27211.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29811.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29811.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29777.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28095.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27648.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27211.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27894.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29777.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29960.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr: 57.6% -- replaced with /var/log/ceph/ceph-client.admin.27648.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28839.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.27894.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29120.log 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.29960.log.gz 2024-09-17T23:33:06.110 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28244.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.28839.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27544.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 54.7% -- replaced with /var/log/ceph/ceph-client.admin.29120.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29847.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.28244.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29716.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.27544.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29366.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29847.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27849.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.29716.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27578.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29366.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29926.log 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27849.log.gz 2024-09-17T23:33:06.111 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29951.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.27578.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27605.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.29926.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27928.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29951.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29540.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27605.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27404.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.27928.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29147.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29540.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29750.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.27404.log.gz 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29531.log 2024-09-17T23:33:06.112 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29147.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29321.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.29750.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30278.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29531.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30022.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29321.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28278.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.30278.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27998.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30022.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28418.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28278.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29049.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 72.8% -- replaced with /var/log/ceph/ceph-client.admin.27998.log.gz 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29741.log 2024-09-17T23:33:06.113 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.28418.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29296.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.29049.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27499.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29741.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29287.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.29296.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30162.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27499.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27367.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29287.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27289.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30162.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27333.log 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.27367.log.gz 2024-09-17T23:33:06.114 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28174.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27289.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27474.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.27333.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29436.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.28174.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29260.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.admin.27474.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27569.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.29436.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28348.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29260.log: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.29260.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28550.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27569.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27779.log 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28348.log: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.28348.log.gz 2024-09-17T23:33:06.115 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28384.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28550.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29637.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27779.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29077.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28384.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29637.log: gzip -5 --verbose -- 18.8% /var/log/ceph/ceph-client.admin.30067.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29637.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29156.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29077.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28479.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.30067.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29181.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.29156.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28516.log 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28479.log.gz 2024-09-17T23:33:06.116 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28690.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29181.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29251.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28516.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28269.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28690.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30171.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29251.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28830.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28269.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28269.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28936.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.30171.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29988.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28830.log.gz 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28199.log 2024-09-17T23:33:06.117 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28936.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29040.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29988.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28735.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28199.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28970.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29040.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28068.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.28735.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28726.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28970.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27718.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.28068.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29856.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28726.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29917.log 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.27718.log.gz 2024-09-17T23:33:06.118 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27169.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.29856.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28586.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29917.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28796.log/var/log/ceph/ceph-client.0.27169.log: 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1%/var/log/ceph/ceph-client.admin.28586.log: -- replaced with /var/log/ceph/ceph-client.0.27169.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27684.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28586.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28769.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28796.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29567.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.27684.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27535.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.28769.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27438.log 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29567.log.gz 2024-09-17T23:33:06.119 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29707.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27535.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28488.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.27438.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27298.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29707.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28559.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.28488.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27955.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.27298.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29820.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.28559.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28235.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27955.log.gz 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27754.log 2024-09-17T23:33:06.120 INFO:teuthology.orchestra.run.smithi139.stderr: 78.3% -- replaced with /var/log/ceph/ceph-client.admin.29820.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29190.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28235.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28866.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.27754.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28445.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.29190.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29890.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28866.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28909.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28445.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28445.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27709.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.29890.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28129.log 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.28909.log.gz 2024-09-17T23:33:06.121 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27745.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27709.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28656.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28129.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27788.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27745.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29461.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28656.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27964.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.27788.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29497.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29461.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27324.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.27964.log.gz 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29086.log 2024-09-17T23:33:06.122 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29497.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29391.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27324.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27395.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.29086.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27824.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29391.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30092.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27395.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28165.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30092.log: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.27824.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28025.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30092.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28165.log: gzip 19.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.28165.log.gz --verbose 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: -- /var/log/ceph/ceph-client.admin.28945.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28760.log 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28025.log.gz 2024-09-17T23:33:06.123 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27639.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.28945.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27429.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28760.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30199.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27639.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28339.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27429.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29680.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30199.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29111.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28339.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30058.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.29680.log.gz 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29881.log 2024-09-17T23:33:06.124 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29111.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29997.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30058.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30137.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29881.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28375.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.29997.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28620.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.30137.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29786.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28375.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28138.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28620.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29671.log 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.29786.log.gz 2024-09-17T23:33:06.125 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28034.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.28138.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27465.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29671.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27508.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.28034.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30031.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27465.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28454.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.27508.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29217.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.30031.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27614.log 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.28454.log.gz 2024-09-17T23:33:06.126 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27989.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29217.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29646.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.27614.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.27989.log: -- /var/log/ceph/ceph-client.admin.29226.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27989.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28665.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.29646.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27919.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28875.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.28665.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.29226.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28979.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 20.6% -- replaced with /var/log/ceph/ceph-client.admin.27919.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29330.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28875.log.gz 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29506.log 2024-09-17T23:33:06.127 INFO:teuthology.orchestra.run.smithi139.stderr: 55.3% -- replaced with /var/log/ceph/ceph-client.admin.28979.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30242.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.29330.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28525.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.29506.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28805.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.30242.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29400.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.28525.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30233.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.28805.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27858.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 51.9% -- replaced with /var/log/ceph/ceph-client.admin.29400.log.gz 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27675.log 2024-09-17T23:33:06.128 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30233.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30128.log 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.27858.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29006.log 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27675.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28104.log 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.30128.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28314.log 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29006.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29576.log 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.admin.28104.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28595.log 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.28314.log.gz 2024-09-17T23:33:06.129 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27358.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28629.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.29576.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27358.log: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.28595.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28409.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27358.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30101.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.28629.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28208.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28409.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30208.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30101.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28900.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.28208.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28059.log 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.30208.log.gz 2024-09-17T23:33:06.130 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27885.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28900.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28699.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28059.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28305.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27885.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29610.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.28699.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.28305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27202.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28305.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29470.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29610.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29015.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27202.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30269.log 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.29470.log.gz 2024-09-17T23:33:06.131 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29427.log 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.30269.log: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.29015.log.gz 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29601.log 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30269.log.gz 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29357.log 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29427.log.gz 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27815.log 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29601.log.gz 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29357.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29357.log.gz 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.27815.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27815.log.gz 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m0.063s 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m0.067s 2024-09-17T23:33:06.132 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.088s 2024-09-17T23:33:06.145 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T23:33:06.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37685.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42723.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35346.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37685.log.gz 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42723.log.gz 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54846.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48990.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35346.log.gz 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54846.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54846.log.gz -5 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43745.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42539.log 2024-09-17T23:33:06.176 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48990.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43745.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42464.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59606.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42539.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59522.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42464.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57568.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59606.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59522.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59522.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35533.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54507.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57568.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57567.log 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35533.log.gz 2024-09-17T23:33:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54671.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54507.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57567.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58919.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36218.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54671.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50141.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58919.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35918.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36218.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50141.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53825.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35918.log.gz 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-09-17T23:33:06.178 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53825.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38903.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53437.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38903.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53222.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53437.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58678.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51378.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53222.log.gz 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-09-17T23:33:06.179 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58678.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51378.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49906.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58052.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43000.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55124.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58052.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45631.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43000.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55124.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58340.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56801.log 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45631.log.gz 2024-09-17T23:33:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58340.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58340.log.gz -5 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57189.log 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57422.log 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56801.log.gz 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46482.log 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57189.log.gz 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58374.log 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57422.log.gz 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46482.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46482.log.gz 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35168.log 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55041.log 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58374.log.gz 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35168.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35168.log.gz -5 2024-09-17T23:33:06.181 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46345.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38256.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55041.log.gz 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46345.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46345.log.gz -5 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55268.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36050.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38256.log.gz 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55268.log.gz 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53612.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36050.log.gz 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35775.log 0.0% 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53612.log.gz 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56393.log 2024-09-17T23:33:06.182 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35775.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35775.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59982.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43234.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56393.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59982.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.59982.log.gz /var/log/ceph/ceph-client.admin.37337.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56894.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43234.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54425.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37337.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56894.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54425.log.gz 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34905.log 2024-09-17T23:33:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56171.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34905.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54323.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56171.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56171.log.gz -5 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38586.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54323.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54323.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57926.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38586.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59396.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47980.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57926.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59396.log.gz 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51699.log 2024-09-17T23:33:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47768.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47980.log.gz 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51699.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51699.log.gz -- 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.45794.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48677.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47768.log.gz 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45794.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.45794.log.gz 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35558.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55061.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48677.log.gz 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35558.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.39219.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35558.log.gz 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58192.log 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55061.log.gz 2024-09-17T23:33:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39219.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.39219.log.gz --verbose 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.41675.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58192.log.gz 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41675.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz -5 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43153.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.34994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36364.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43153.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.50713.log -- replaced with /var/log/ceph/ceph-client.admin.43153.log.gz 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48364.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36364.log.gz 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55717.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50713.log.gz 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41955.log 2024-09-17T23:33:06.186 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48364.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55717.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41955.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51542.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz -5 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36260.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36879.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51542.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46757.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36260.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44433.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36879.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46757.log.gz 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35856.log 2024-09-17T23:33:06.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57632.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44433.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35856.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35796.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57632.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35796.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45088.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37074.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45088.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44259.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37074.log.gz 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39949.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44259.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T23:33:06.188 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44259.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41122.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39949.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52097.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41122.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46798.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51163.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52097.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52426.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51163.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59942.log 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52426.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44053.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T23:33:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44155.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50276.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59942.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41102.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44155.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50276.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50276.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53510.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57484.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41102.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53510.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37216.log 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57484.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-09-17T23:33:06.190 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42823.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54246.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37216.log.gz 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42823.log.gz 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58968.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54246.log.gz 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54724.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58968.log.gz 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54724.log.gz 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48253.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58403.log 2024-09-17T23:33:06.191 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48253.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48253.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46563.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58850.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58403.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46563.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57249.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39743.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58850.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56466.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57249.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39743.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56466.log.gz 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35564.log 2024-09-17T23:33:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35348.log 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53785.log 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35564.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35348.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55166.log 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53785.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40528.log 0.0% 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55166.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40528.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.51679.log -- replaced with /var/log/ceph/ceph-client.admin.40528.log.gz 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59312.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44279.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51679.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59480.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59312.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41831.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.44279.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52998.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59480.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59564.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41831.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52823.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52998.log.gz 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35126.log 2024-09-17T23:33:06.194 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59564.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37708.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52823.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35350.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35126.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51903.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37708.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46028.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35350.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37021.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51903.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57758.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46028.log.gz 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58466.log 2024-09-17T23:33:06.195 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37021.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36176.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57758.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53869.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58466.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36176.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36176.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58734.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53869.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59186.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58734.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37584.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51236.log 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59186.log.gz 2024-09-17T23:33:06.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37584.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52251.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51236.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52251.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37921.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35351.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52548.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37921.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35351.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37983.log 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52548.log.gz 2024-09-17T23:33:06.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50733.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37983.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43489.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.50733.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35624.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50733.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56567.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43489.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35624.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35624.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55789.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57016.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56567.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39905.log 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55789.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57016.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.57016.log.gz 2024-09-17T23:33:06.198 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35620.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59816.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39905.log.gz 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35620.log.gz 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36484.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35050.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59816.log.gz 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58920.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36484.log.gz 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35050.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35050.log.gz 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58187.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58194.log 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58920.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58920.log.gz 2024-09-17T23:33:06.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58187.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53908.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58010.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58194.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59354.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53908.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58189.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58010.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40513.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59354.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39461.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58189.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40513.log.gz 2024-09-17T23:33:06.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39461.log.gz 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57800.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36092.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50551.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57800.log.gz 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58738.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36092.log.gz 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53160.log 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50551.log.gz 2024-09-17T23:33:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32319.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58738.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.53160.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53160.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.32319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55442.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32319.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49497.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55442.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49497.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38216.log 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-09-17T23:33:06.202 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38216.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38216.log.gz 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59690.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57842.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59690.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59690.log.gz 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42149.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57842.log.gz 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57333.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42149.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42149.log.gz 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48020.log 2024-09-17T23:33:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57333.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48020.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48020.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49356.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39648.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42878.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49356.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.39648.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40223.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39648.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42878.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54206.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40223.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40223.log.gz 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58191.log 2024-09-17T23:33:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58191.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58191.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35509.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49263.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36746.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35509.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.49263.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59228.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49263.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59438.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36746.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55483.log 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59228.log: /var/log/ceph/ceph-client.admin.59438.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59438.log.gz 2024-09-17T23:33:06.205 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50887.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59228.log.gz 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55483.log.gz 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51575.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50887.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50887.log.gz 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45753.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51575.log.gz 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45753.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45753.log.gz 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59011.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43490.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59732.log 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-09-17T23:33:06.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59011.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49731.log 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36485.log 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59732.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49731.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49731.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53080.log 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36485.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40754.log 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53080.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51502.log 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.40754.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59774.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40754.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51502.log.gz 2024-09-17T23:33:06.207 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58618.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59774.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59774.log.gz 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45436.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58618.log.gz 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58677.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40827.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45436.log.gz 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.58677.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58188.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58677.log.gz 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40827.log: gzip/var/log/ceph/ceph-client.admin.58188.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32016.log 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40827.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58188.log.gz 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.208 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.32016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59270.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32016.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44556.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59270.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44516.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44556.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44556.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55790.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44516.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27633.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55790.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55790.log.gz 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57464.log 2024-09-17T23:33:06.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.27633.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27633.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52916.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57464.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35776.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52916.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54999.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52016.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35776.log: /var/log/ceph/ceph-client.admin.54999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41788.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54999.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35776.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52016.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54104.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41788.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41788.log.gz 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.58406.log 2024-09-17T23:33:06.210 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54104.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39393.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58406.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58406.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56118.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39393.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59648.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56118.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56118.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38727.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59648.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43388.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38727.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36487.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49926.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43388.log: 0.0%gzip/var/log/ceph/ceph-client.admin.36487.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58464.log 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43388.log.gz 2024-09-17T23:33:06.211 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36487.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55893.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49926.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39016.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58464.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35352.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55893.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39016.log: -- /var/log/ceph/ceph-client.admin.52721.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39016.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35352.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54205.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52721.log: gzip/var/log/ceph/ceph-client.admin.54205.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35075.log 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54205.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52721.log.gz 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39825.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35075.log: /var/log/ceph/ceph-client.admin.50223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59144.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35075.log.gz 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39825.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39825.log.gz 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59144.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59144.log.gz 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40928.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59102.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40928.log: /var/log/ceph/ceph-client.admin.36134.log 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40928.log.gz/var/log/ceph/ceph-client.admin.59102.log: 2024-09-17T23:33:06.213 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59102.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54609.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36134.log: /var/log/ceph/ceph-client.admin.36008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57884.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36134.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54609.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57884.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38390.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.32245.log: -- /var/log/ceph/ceph-client.admin.50948.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38390.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35896.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57375.log 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50948.log.gz 2024-09-17T23:33:06.214 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36302.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.35896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35896.log.gz 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43960.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57375.log: /var/log/ceph/ceph-client.admin.36302.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59858.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57375.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36302.log.gz 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43960.log.gz 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40457.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59858.log.gz 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58598.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40457.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40457.log.gz 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57968.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58598.log.gz 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42424.log 2024-09-17T23:33:06.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57968.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57968.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57674.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42424.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39588.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59900.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57674.log: /var/log/ceph/ceph-client.admin.39588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41440.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57674.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39588.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59900.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59900.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41440.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41440.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.41420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47032.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57716.log 2024-09-17T23:33:06.216 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46081.log 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38472.log 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47032.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57716.log.gz 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58870.log 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46081.log.gz 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38472.log.gz 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58870.log.gz 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55974.log 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-09-17T23:33:06.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55974.log.gz 2024-09-17T23:33:06.218 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T23:33:06.337 INFO:teuthology.orchestra.run.smithi031.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T23:33:06.932 INFO:teuthology.orchestra.run.smithi031.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T23:33:08.741 INFO:teuthology.orchestra.run.smithi031.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T23:33:09.099 INFO:teuthology.orchestra.run.smithi037.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T23:33:12.164 INFO:teuthology.orchestra.run.smithi037.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T23:33:16.149 INFO:teuthology.orchestra.run.smithi037.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T23:33:16.151 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-17T23:33:16.151 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m10.185s 2024-09-17T23:33:16.151 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m19.046s 2024-09-17T23:33:16.151 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.692s 2024-09-17T23:33:17.834 INFO:teuthology.orchestra.run.smithi031.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T23:33:17.836 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-17T23:33:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m11.871s 2024-09-17T23:33:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m15.468s 2024-09-17T23:33:17.836 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.742s 2024-09-17T23:33:17.837 INFO:tasks.ceph:Archiving logs... 2024-09-17T23:33:17.837 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi031/log 2024-09-17T23:33:17.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T23:33:19.926 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi037/log 2024-09-17T23:33:19.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T23:33:22.075 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi139/log 2024-09-17T23:33:22.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T23:33:22.238 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T23:33:22.257 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T23:33:22.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T23:33:22.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T23:33:22.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T23:33:22.346 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T23:33:22.358 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T23:33:22.358 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:33:22.361 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:33:22.364 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T23:33:22.377 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:33:22.379 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:33:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-17T23:33:22.381 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:33:22.382 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-17T23:33:22.382 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 -188us[ -188us] +/- 41ms 2024-09-17T23:33:22.382 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -721us[ -720us] +/- 74ms 2024-09-17T23:33:22.382 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +1161us[+1161us] +/- 79ms 2024-09-17T23:33:22.382 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:33:22.384 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:33:22.384 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-17T23:33:22.384 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -177us[ -174us] +/- 41ms 2024-09-17T23:33:22.384 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -729us[ -726us] +/- 74ms 2024-09-17T23:33:22.384 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +1129us[+1129us] +/- 79ms 2024-09-17T23:33:22.384 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:33:22.386 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T23:33:22.386 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-17T23:33:22.386 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -164us[ -154us] +/- 41ms 2024-09-17T23:33:22.386 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -713us[ -713us] +/- 74ms 2024-09-17T23:33:22.386 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +1156us[+1156us] +/- 79ms 2024-09-17T23:33:22.386 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T23:33:22.386 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T23:33:22.394 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T23:33:22.395 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T23:33:22.404 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T23:33:22.512 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T23:33:22.594 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T23:33:22.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:33:22.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:33:22.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:33:22.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:33:22.724 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T23:33:22.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:33:22.754 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T23:33:22.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T23:33:22.780 INFO:teuthology.task.internal:Duration was 680.786847 seconds 2024-09-17T23:33:22.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T23:33:22.827 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T23:33:22.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T23:33:22.830 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T23:33:22.832 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T23:33:22.872 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:33:22.873 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:33:22.874 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T23:33:23.287 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T23:33:23.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-17T23:33:23.287 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T23:33:23.311 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-09-17T23:33:23.312 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T23:33:23.356 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-17T23:33:23.357 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T23:33:23.393 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T23:33:23.393 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T23:33:23.396 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T23:33:23.399 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T23:33:23.464 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T23:33:23.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T23:33:23.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T23:33:23.471 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T23:33:23.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T23:33:23.622 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T23:33:23.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T23:33:23.648 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T23:33:23.674 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T23:33:23.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T23:33:23.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:33:23.712 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:33:23.716 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:33:23.733 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-17T23:33:23.738 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-09-17T23:33:23.763 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-17T23:33:23.811 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:33:23.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:33:23.826 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:33:23.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:33:23.841 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T23:33:23.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T23:33:23.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T23:33:23.875 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T23:33:23.876 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi031 2024-09-17T23:33:23.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T23:33:23.913 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi037 2024-09-17T23:33:23.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T23:33:23.948 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909589/remote/smithi139 2024-09-17T23:33:23.949 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T23:33:23.986 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T23:33:23.987 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T23:33:23.989 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T23:33:23.992 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T23:33:24.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T23:33:24.046 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T23:33:24.046 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T23:33:24.055 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T23:33:24.055 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T23:33:24.058 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T23:33:24.060 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T23:33:24.071 INFO:teuthology.orchestra.run.smithi031.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 23:33 /home/ubuntu/cephtest 2024-09-17T23:33:24.072 INFO:teuthology.orchestra.run.smithi037.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 23:33 /home/ubuntu/cephtest 2024-09-17T23:33:24.088 INFO:teuthology.orchestra.run.smithi139.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 23:33 /home/ubuntu/cephtest 2024-09-17T23:33:24.089 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T23:33:24.099 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T23:33:24.122 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 680.7868473529816 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T23:33:24.123 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T23:33:24.191 INFO:teuthology.run:pass